Cyclos reference documentation

This method will deploy Cyclos as a web application of an existing Apache Tomcat server.

There is a Docker image for Cyclos, and the installation via Docker is very easy, and can be accomplished with a few steps. It can also be used for production with no drawbacks when compared to the Tomcat installation.

For details on how to install Cyclos via Docker image, visit the Cyclos repository on Docker hub. It contains detailed information on installation and maintenance.

For details on how to install docker, please visit https://docs.docker.com/engine/install/.

This method will deploy Cyclos in Amazon Elastic Compute Cloud (EC2). Amazon Web Services (AWS) is a cloud computing platform that provides a wide range of services, including computing power, storage, and databases. EC2 is a web service that provides resizable compute capacity in the cloud.

In this guide we will an Auto Scaling group to run Cyclos. Cyclos uses Hazelcast for clustering. In the following steps, we will follow the instructions to deploy Cyclos in an EC2 instance with Hazelcast. The database will be hosted externally, in an RDS instance.

Please, note that there are many steps to follow. Please, follow each one carefully!

This method will deploy Cyclos with Kubernetes using Amazon Elastic Kubernetes Service (EKS). Kubernetes is the industry standard system for cluster orchestration, but the setup process is provider-dependent. Many of the following concepts and steps can be adjusted for other providers.

Please, note that there are many steps to follow. Please, follow each one carefully!

The default memory heap size of Tomcat is very low. You can augment this in the following way:

Windows

In the bin directory of Tomcat create (if it doesn't exist) a file called setenv.bat, edit this file and add the following line:

Linux

In the bin directory of Tomcat create (if it doesn't exist) a file called setenv.sh, edit this file and add the following line:

When an OutOfMemoryError (OOME) is unhandled, it is generally advisable to kill the JVM. Although any operation can, theoretically, generate `OOME’s, one particular one is prone to it: generating PDFs. Cyclos uses the excellent openhtmltopdf library, which transforms HTML documents to PDFs. However, as most HTML handling applications (such as browsers), the RAM requirement is high, depending on the size of the HTML document being processed.

When generating PDFs, Cyclos attempts to catch `OOME’s. However, if some other Tomcat thread gets out of memory in the same time, the entire server will become unresponsive, and there will be no additional choice than restarting the server.

The recommended approach is to use pass a JVM property (in a similar fashion to memory adjustments) to kill the JVM process in such cases. It is then the responsibility of an external service monitoring tool to restart the Tomcat server. The parameter to pass is -XX:OnOutOfMemoryError="kill -9 %p".

In most systems, Cyclos is either:

In both cases, if the Tomcat server has an unhandled OOME, it will be restarted instead of becoming unresponsive.

Enabling SSL is crucial on live systems, as it protects sensitive information, like passwords, to be sent plain over the Internet, making it readable by eavesdroppers.

Generally it is advised to use a proxy server, like Apache or Nginx, that handles HTTPS and then redirect the request to Tomcat. See Enable SSL on Apache for more details.

Otherwise, if the Tomcat server is directly accessible from the Internet, to enable SSL / HTTPS you first have to enable (uncomment) the https connector in the file <tomcat_home>/conf/server.xml

Generate a key with the keytool utility that comes bundled with Java:

After executing this command, you will first be prompted for the keystore password. Passwords are case-sensitive. You will also need to specify this password in the server.xml configuration file, as described later.

Next, you will be prompted for general information about this certificate, such as company, contact name, and so on. This information will be displayed to users who attempt to access a secure page in your application, so make sure that the information provided here matches what they will expect.

Finally, you will be prompted for the key password, which is the password specifically for this certificate (as opposed to any other Certificates stored in the same keystore file). You MUST use the same password here as was used for the keystore password itself, or Tomcat will have problems loading it. Currently, the keytool prompt will tell you that pressing the ENTER key does this for you automatically.

If everything was successful, you now have a keystore file with a certificate that can be used by your server.

Clustering is useful both for scaling (serving more requests) and for high availability (if a server crashes, the service continues to run). Please note that when upgrading Cyclos to a newer version the entire cluster must be shut down which will cause some downtime.

There's no need to configure a Tomcat-level cluster, because it is only used to replicate HTTP sessions. Cyclos, however, doesn't use Tomcat sessions, but handles them internally. This way, there is no special Tomcat configuration to support a Cyclos cluster.

The Cyclos application, however, needs some small configurations to enable clustering. Cyclos uses Hazelcast to synchronize shared state (such as caches) between cluster hosts. To enable clustering, find in cyclos.properties the line containing cyclos.clusterHandler, and set it to hazelcast.

Hazelcast is able to find the cluster nodes with many different join strategies: multicast (for local network), TCP/IP (when the list of nodes is known beforehand), Kubernetes (using a DNS service), Amazon Web Services (by matching nodes via tags) and Google Cloud Platform (by matching nodes via labels). Starting with Cyclos 4.16, it is possible to both enable clustering and configuring the join strategy by setting one of these environment variables:

Another way to configure the join strategy, as well as fine-tuning other parameters, is by configuring the WEB-INF/classes/hazelcast.xml file. Check the Hazelcast documentation for more details. When one of the previously mentioned environment variables are found, any <join> tags in hazelcast.xml are ignored, and the environment variable is used instead.

To set up high-availability at database (PostgreSQL) level, it is recommended to either use a managed PostgreSQL service from a cloud provider or deploy a PostgreSQL cluster with CloudNativePG.

You can use Apache as a front-end / load balancer for Tomcat. This is very useful when you have several domains configured on the server. There are several documentations and examples available on the Internet. In our example, we will use the module mod_jk.

The configuration is done in the /etc/libapache2-mod-jk/workers.properties file. By default, this is configured to use the AJP port 8009, this is the default AJP port for Tomcat, if you are using a different port you need to configure it here.

On Tomcat, we need to enable the AJP connector. Edit the file <tomcat_home>/conf/server.xml and uncomment the AJP connector:

Now on Apache we need to configure the virtualhost (which on Ubuntu can be found here in /etc/apache2/sites-enabled/) to use the AJP connector. On the virtualhost of your domain, add the following lines:

This example uses the cyclos as ROOT application on Tomcat. If you want to use something like http://www.yourdomain.com/instance_name we need to deploy cyclos on the webapps/instance_name` directory and configure apache like this:

Now restart both Apache and Tomcat and check if it works.

Enabling SSL is crucial for live systems, as it protects sensitive information, like passwords, to be sent plain over the Internet, making it readable by eavesdroppers.

We recommend using Let’s Encrypt to set up a certificate. It is free and has only a few steps for setup.

The easiest configuration for a load balancer is Apache connecting to Tomcat using the AJP protocol. In this case, the original request is forwarded to Tomcat as is, keeping the original client IP address and URL.

However, in most other cases, the load balancer works as a proxy, sending a new HTTP to Tomcat and forwarding the response to the client. Examples of such proxies include Apache with mod_proxy, Nginx, haproxy and all cloud provider load balancers.

In either way, generally the proxy will have the server certificate and will terminate the SSL connection with the client. Then an internal (non-HTTPS) request is performed from the proxy to Tomcat.

That means the client IP address received by Cyclos, as well as the request URL, are different from the original request performed by the client. As Cyclos uses the client IP for logging and blocking in case of abuse, this would lead Cyclos to block the proxy, preventing any further request.

NOTICE: There is an important security consideration when using a proxy. See below the cyclos.header.remoteAddress.index setting for more details.

However, the proxy will add some extra request headers, with information about the original request. Cyclos then needs to be configured to read both the IP address and which was the connection protocol used by the original request (HTTP or HTTPS) from those headers, instead of directly from the incoming HTTP request. For this, the following settings in cyclos.properties are needed:

The following cases are handled by Cyclos to match a specific network / configuration from the request URL:

When Tomcat is behind a proxy, it will never receive requests using the original public URL. Hence, only matching by network (and optionally, configuration paths) will be used.

Still, networks should correctly set the root url in their configurations because they are used to generating full URLs at the server side, for example, when sending links in e-mails or resolving image URLs in rich texts.

Following are common cases that could be configured for a proxy, all assuming Cyclos is deployed to a Tomcat accessible by the proxy via http://tomcat:8080/instance_name:

There are reserved paths in Cyclos that cannot be used as paths in proxies. For example, a proxy could handle requests to https://www.my-project.com/app and redirect them to http://tomcat:8080/instance_name. In this case, the /app path part is public, used by clients, but never visible on Tomcat.

To handle such cases, the list of reserved paths is used to generate the correct URIs for scripts and stylesheets, and having any of the reserved paths in the proxy would prevent the URI generation from working correctly.

The list of reserved paths is:

Cyclos supports displaying maps using Google Maps. This has to be enabled in the Cyclos configuration. Cyclos also uses geocoding to map the user-informed address fields to a position (latitude/longitude).

Google Maps requires an API key. For details on the free daily quota for map views and geocode requests, see this page.

There are actually 2 API keys that can be set in the Cyclos configuration: The server-side API key and the browser API-key.

Each one needs to be generated in the API Manager.

Once the API keys are ready, they can be copied / pasted into the Cyclos configuration, on the corresponding "Google maps server API" and "Google maps browser API" fields.

The API Manager allows monitoring of requests performed by each API.

On the main Cyclos web application, addresses are geocoded (having the address converted to a latitude / longitude coordinates) on the client-side, before saving it. In such cases, the browser API key is used. However, sometimes addresses can be saved without being geocoded, either by third-party software or by importing new users into Cyclos. In such cases, a server-side background task will attempt to geocode them (might take up to 24 hours for this) using the server API key.

If the server-side geocode fails, the address is permanently marked in the database as 'failed', so it won’t be indefinitely reattempted. When saving the address again, the address failure will be cleared out, so it will be attempted again. However, in some cases it might be desired to reattempt the geocoding for all stored addresses, for example, when replacing an API key that was invalid. In such cases, the administrator can run the following SQL query in the database:

Afterwards, under Reports > System information > Recurring tasks, run the 'Address geocoding' task, which will start the process right away.

Starting with Cyclos 4.15, it has been added the support for reCAPTCHA v2, which displays the "I am not a robot" checkbox. As of 2021, it offers a free usage for up to 1 million requests per month. As CAPTCHAS are used in Cyclos only on registration and on 'forgot password' requests, it should be enough for most systems.

To enable it, first you need to register a new site in reCAPTCHA. Select the v2 on reCAPTCHA type. Then add your domain, accept the terms of service and submit. It will display 2 keys: the site key and the secret key.

Then, in Cyclos configuration form, under the CAPTCHA section, select reCAPTCHA v2 as provider and paste both keys to the corresponding fields.

IMPORTANT! To make the reCAPTCHA works for the mobile app, please ensure the web server doesn't send the following HTTP response headers for the path /mobile-recaptcha:

Cyclos handles stored files, such as images, documents and binary custom field values. By default, they are stored in the database. However, for large systems, having files in the database will complicate backups, which will be very large. Also, having files in the database can be suboptimal in terms of performance. However, Cyclos also provides other storage types.

When a PostgreSQL hot standby server is used, the master server can be offloaded by directing read-only queries to the hot standby server. This can be obtained by specifying additional datasource properties, with the cyclos.datasource.readOnly prefix. For example:

All properties specified for the regular datasource can also be specified for the read-only datasource. The default value of all read-only datasource properties are those set for the regular datasource. You can then override only the ones that are different. So, in this example, the same Hikari dataSource.databaseName, dataSource.user, and so on, will be used for both data sources.

An important point for consistent queries between the master and the replica servers is the synchronous_commit setting in the master server. It should be set to either on (the default) or remote_apply. Take care that the setting is only used when synchronous_standby_names is also set. The remote_apply setting will wait until the data is visible for queries in the replica before the commit ends. This will introduce an additional delay in read-write transactions, but ensures data consistency. Without it, it might happen that after saving some data, the next request to read the same data will fail. This is a trade-off between consistency and performance. However, the odds for this are low, because there are the delays between the first request’s response being read by the client and the second request being sent and processed by the server. It is very like that such times are higher than the replica server time to apply the changes. This policy should be defined by the system administrator.

For large systems, the time required for searching users, advertisements or transactions can be unacceptable when searching the database. Such queries can be complex, using full-text keywords or geo-distance filters.

For such systems, it is advised to use OpenSearch as search provider. For details on how to configure Cyclos with OpenSearch, refer to this section.

By default, Cyclos logs access to services, as well as background tasks, to files. But besides logging to files, it is also possible to log to an external PostgreSQL database. Logging to an external database has some benefits, especially being easier to find data when needed.

Logs are, by default, asynchronous, so the requests are not delayed until the log is written. This is controlled by the cyclos.log.threads property, which sets the number of threads used to concurrently write logs. However, if the server crashes, some log entries may be lost. If the number of threads is set to zero, logs will be synchronous, delaying each response. This guarantees that all logs are written before returning the response, but can have a high impact on the system throughput.

To choose the log provider, the cyclos.log setting should be changed in cyclos.properties. There are also additional settings for specific log providers:

Logging to files

Logging to an external database

It is important that the PostgreSQL database used for storing logs isn’t the same as the main Cyclos database. The database itself must be created before starting Cyclos.

If using a database log, make sure to set the maximum number of connections equal to the number of threads set in the cyclos.log.threads setting. When using synchronous logging, the maximum connections to the log database will also limit the number of concurrent requests, so, extra care is needed.

A final note on the log database: there are currently 2 tables: service_logs and task_logs. The service_logs store a row each time a client calls any Cyclos service, while the task_logs stores a row for each background task execution. To make insertion of rows as fast as possible, the tables have no indexes, and will store parameters and results as the PostgreSQL’s JSON type, which has minimal impact on inserts (comparing to JSONB).

However, for searching data, the JSONB type (binary JSON) is more efficient, and supports indexing. When searching the table with too many logs, instead of searching directly on service_logs, it is recommended to create a new table, and query it instead. This new table should have the same columns as service_logs, but with JSONB columns instead. Also, add indexes according to your query.

A final note on log tables is that they tend to quickly grow in size, so you may need to periodically (according to the database data volume) move old data to another database in order to not impact the logging performance. Also, don’t forget to vacuum the table after deleting old records.

For instructions on how to host the frontend separately from Cyclos please see the project instructions page at GitHub.

Very large systems may produce a huge amount of transactions per day. For such systems, having the full data over many years in the production database may be challenging, as it increases the database space, makes it harder to backup the database, increases the OpenSearch index size, etc.

Starting with Cyclos 4.16, such systems can define a data retention period - for example, 2 years - and configure Cyclos to only consider transfers, transactions and balances within this period. Then, an external application (provided by STRO upon request) connects to the Cyclos database, backups data in an external database, and then physically deletes such data in the Cyclos production database.

Cyclos has an integrated feature that allows administrators (with permissions) directly in the account history page to search for archived transactions of that account.

For understanding which data is archived up, it is important to understand the structure of transfers vs transactions in Cyclos. A quick review: a transfer is an actual balance transfer between 2 accounts, while a transaction is an intent of transferring balances between accounts. There are 2 kinds of transactions: payments, which generate one or more transfers once processed, and others which generate a payment (that will generate transfers). Payments are of 3 kinds: direct, which generate a single transfer once processed; scheduled, which splits the total amount in one or more installments (and each installment generates a transfer when processed); and recurring, which repeats the same amount on every processed occurrence (generating a transfer per occurrence). Other kinds of transactions are the ones which generate a payment when processed: payment requests, tickets and external payments.

These are the data effectively copied to the archive:

Carefully consider that the following data will be permanently lost after archiving:

Note: When physically removing transfers after archiving, it is then possible to permanently remove transfer types or transaction custom fields which were only referenced from archived transfers. However, this is not recommended, because the same data will be used to show the details of the archived transfer in Cyclos.

In order to enable archiving, you have to set the following in cyclos.properties:

In OpenSearch, deleting many documents is a heavy operation, causing a high resource (CPU / memory) utilization. This impacts queries while documents are being deleted. Cyclos adopts a throttling strategy by deleting batches of documents with a wait time between each request. However, this should only be done during low-traffic times, because even throttled deletes slow down queries. The following settings control this behavior:

Also, Cyclos has a permission which can be enabled in administrator permissions / products (only visible when archiving is enabled in cyclos.properties) for administrators to query archived data. This functionality is available from the account history page itself, with a button to query for archived data, and fetches data from the external application via API.

This archiving functionality should not be confused with specific archiving settings in cyclos.properties, which are: cyclos.purgeMessagesOnTrash.days, cyclos.archiveImports.days, cyclos.archiveAccountFees.days and cyclos.archiveBulkActions.days. Those settings control the plain removal of data periodically from the Cyclos database.

Archiving is performed in a monthly basis. A recurring task will check daily whether the current month’s archiving was performed. If not (for example, the server was down on the first day), it will calculate the new archiving date and update the archived balances on all accounts. After all accounts archived balance are calculated, Cyclos will notify the archiving application to start the external archive.

When enabled, information about the current archiving status can be seen from the system monitor page.

All data in Cyclos is stored in the PostgreSQL database. Making a backup of the database can be done using the pg_dump command. In terms of files, you only need to backup the cyclos.properties and, if customized, the hazelcast.xml configuration files.

The database can be backed up manually as follows:

Note: in this example the name of the database is cyclos4, the username cyclos and the command will prompt for the password of the cyclos user.

To restore a database dump to another database, first create the new database (in this example, cyclos4) and grant permission to the user to manage it (in this example, the user is cyclos). Supposing the file containing the dump is cyclos4.sql in the current directory, the following command will import the data:

When the database is very large (especially if it has a lot of images) it is possible to use a custom format for the dump file, which makes the dump file smaller. To use it, backup with the following command:

To restore the dump, another command needs to be used as well:

Note: for larger databases, it is highly advised to store binary files outside the database, specially in a managed storage service such as Amazon S3 or Google Cloud Storage. For more details, see External content storage.

If you lost the password of your global administrator, it is still possible to update the value on the database directly. To reset the password to 1234, run the following sql in the PostgreSQL query tool (psql). Replace the 'admin' username by the username of your global administrator.

If you are locked out of the system because an applied theme is failing compilation because of some error, you can run the following SQL to apply a built-in theme to both guests and logged users. After running it, only the global default URL will have an applied theme - all others will just inherit them.

If Cyclos development team or a third party asks you to share the database with them, specially for bug isolation and support, it is vital for security that sensitive configurations, passwords and personal data are removed from the database.

The passwords in Cyclos are hashed with one of the strongest algorithms available (Bcrypt), but still passwords can be theoretically recovered using brute force (although very unlikely). If the database falls into the wrong hands, some users might get compromised. Therefore, it is always recommended to follow this procedure before sharing the database with other parties:

Reset all passwords to '1234':

Remove sensitive user information:

If you have a custom field with private information, such as ID card, you can also remove the values with (adjust the <internal_name> value):

Remove sensitive configurations:

You may also have API keys, passwords, etc. stored in custom script parameters. If this is the case, make sure to also change that.

Finally, create a dump of the temporary database. This dump and then be sent to the third party.

For running a production system, it’s of vital importance to run not only a live instance but also, a test instance. Besides running a test instance, many systems will benefit from also having a development instance. When updating Cyclos, creating new scripts, or doing any change with impact to the system, we recommend to always try it out on the test instance first. If all goes well on the test system, the change can be made on the live system too.

We recommend creating a script that copies the database from a live instance backup and, before the test instance is started, imports that dump and removes all vital information from the database. It’s very important that customers won’t receive emails, SMS messages or mobile app push notifications from a test instance.

We recommend first removing all user-related private information and production-specific API keys. For this, please refer to the Share database dumps section.

Also, the following database commands are recommended, and can be adjusted to better fit your needs. Adjust the variables presented between < and >.

Finally, we recommend using a test SMTP server, in this way a client could never accidentally get an email message. An easy and free solution for this is Mailtrap, but there are also self-hosted solutions available, such as Mailhog. Here’s an example for Mailtrap:

After all these adjustments, you can start the test instance.

A common practice for a first-time configuration of Cyclos, specially with a complex structure for accounts, configurations, products and groups, is to configure all the system, and create some test users and payments. However, after finishing configuration, it might be desirable to remove all users and transfers (payments) from that network, leaving only administrators and configurations. Alternatively, it might be desirable to completely delete an entire network.

An interactive utility is included in Cyclos, which can be used for both cases. Please, be advised to perform a full database dump before running the utility, and have Cyclos stopped before running it.

To run the utility, go to <tomcat_home>/webapps/<instance_name> directory and execute the following:

If Tomcat’s lib directory is located elsewhere, replace the :../../lib/*: part with :/path/to/tomcat/lib/*:. If that’s the case, you should see an error like java.lang.ClassNotFoundException: javax.servlet.ServletContext.

Then follow the instructions presented on the console. When a lot of data is removed, it is recommended to run a full vacuum in the database. This operation might take a while. To run it, execute the following command:

If you are experiencing specific performance issues, it might be useful to profile the system to find the bottlenecks, especially when using scripts. Starting with Cyclos 4.16.12, global system administrators have the Reports > System profiling menu. In this menu you can start a profiling session. Then, until manually stopped, the system will collect data about the time spent in several entry-points:

The procedure to collect metrics is:

If you have a support contract with STRO, the team can help you analyzing the data and identifying the bottlenecks.

Due to a (small) performance impact in the runtime while profiling is running, it is not recommended to leave it running for long periods. Also, more memory will be used to store the collected data.

In cyclos.properties there is a setting cyclos.profiling.maxEntries which limits the number of entries collected. The default is 10000. If more entries are collected, older are discarded. Note that if you are running in a cluster, this applies per cluster node. So, the actually downloaded data may contain up to number_of_nodes * cyclos.profiling.maxEntries entries.

It is also possible to apply other filters to what is collected, if you know what you need to profile:

In future Cyclos versions, a frontend will be developed to analyze the collected data, so visualizing the bottlenecks will be easier.

In this mode, a principal (user identification method), which can be the login name, e-mail, mobile phone, custom field, account number or token value (card number), depending on the channel configuration, is sent on each request together with the password (live systems must always be over HTTPS, so should be secure). The drawbacks are that the username and password need to be stored in the client application, and changing the password on the web (if the same password type is used) will make the application stop working.

In this mode, first a request authenticated with user and password is made with POST /api/auth/session. It returns a session token, as well as other information of the logged user. Subsequent requests should pass this session token instead in the subsequent requests, via the Session-Token HTTP header. To end a session (logout), a request authenticated with the session token must be performed with DELETE /api/auth/session.

Access clients can be configured to prevent the login name and password to be passed on every request by clients, decoupling them from the actual password, which can then be changed without affecting the client access. It also improves security, as each client application has its own authorization token, which can be individually blocked or revoked.

To configure access clients, first a new identification method of this type must be created by administrators in 'System > System configuration > User identification methods'. Then, in a member product of users which can use this kind of access, permissions over that type should be granted. Finally, the user (or an admin) should create a new access client in Cyclos main access, and get the activation code for it.

The activation code is a short (4 digits) code which uniquely identifies an access client pending activation for a given user. To use the access client, on the client application side (probably a server-side application or an interactive application), a request must be performed: POST /api/clients/activate, passing the username / password in a BASIC authentication and sending the activation code as the code query parameter.

The result will be a token which should be passed in subsequent requests using the HTTP header Access-Client-Token. The activation process should be done only once, and the token will be valid until the access client in Cyclos is blocked or disabled.

Here is an example which can be called by the command-line program curl:

The generated token will be printed on the console, and should be stored on the client application to be used on requests.

Additionally, clients can improve security if they can have some unique identifier which can be relied on, and don’t need to be stored. For example, Android devices always provide a unique device identifier. In that case, this identification string can be passed at the moment of activation, and will be stored on the server as a prefix to the generated token. The server will return only the generated token part, and this prefix should be passed on requests together with the generated token. The prefix is passed in the activation request as a query parameter prefix. So, for example:

Imagining the server returns the fictional token ABCDEFG (the actual token is 64 characters long), the actual token that then should be passed to requests is XYZWABCDFG.

Scripts are executed in a database transaction. If there are any exceptions, the transaction is rolled-back. Otherwise, if all operations succeed, the transaction is committed. Scripts must never modify the current transaction to avoid database inconsistencies. This is a very hard situation to fix, requiring manual intervention in the database, and the Cyclos team cannot be held responsible in such cases.

Below are situations which can cause database inconsistencies.

Do not make any HTTP requests from scripts to Cyclos' own REST operations or custom web services. Though the REST services are well documented and easy to use, doing so will require opening a separated and parallel database transaction to handle the request, while leaving open the current transaction which is running the script. This can easily exhaust the database connection pool and even lead to situations where no more requests can be handled. Instead, scripts should only use the internal services and handlers.

Do not create your own ExecutorService or ScheduledExecutorService unless you have a very good reason for it. Instead, used the shared scheduled executor which is available through the invokerHandler bounded variable, using its executorService property. It scales and shuts down threads as needed. Here is an example:

Keep in mind that Groovy will resolve closures as Runnable by default, not as Callable. This means that the result of the Future.get() call will always be null, as the submit(Runnable) version is called in runtime, which always returns null. To actually use the result, you need to explicitly cast your closure to Callable, like this:

However, if you really need to create an ExecutorService, make sure to shut it down before the script ends. Failing to do so will cause the threads in the pool to be left open (depending on the executor type), which increases the number of system threads in each script execution. This will eventually crash the JVM due to an excessively large number of threads. So, enclose your code with a try / finally block and call ExecutorService.shutdown().

It is not a good idea to have a large job running in the same transaction. This is particularly dangerous for payments because each payment requires locking accounts*, and locks are only released on commit or rollback, causing the locks to be held for much time.

Instead, it is much better to break the job into smaller pieces. For example, you could move payments out of the main transaction, by using :

With this approach, locks will be released early, as the only thing that the separated transaction does is the payment, which would release the lock early.

Also, if you need to do some processing iterating a large amount of data (e.g. users, records, etc.) take a look at the custom background tasks, it is a much better option than manually processing each entity in the same transaction.

* How the account locks are performed can be changed through the payment type being used, please check the information for the "Lock origin account" and "Lock destination account" properties on the transfer type details page.

Groovy provides the Grape functionality to fetch Java dependencies in a Maven repository. All you need is to find the dependency in https://mvnrepository.com/, click on the version and copy the content of the Grape tab.

Working directly with custom field values can be a hard task. For this reason, Cyclos provides the scriptHelper.wrap(object[, customValues]) method. It will return a Map<String, Object> which can be used to set and get custom field values. When setting a value, the input will be converted the value to the expected type, whereas when reading a value, the 'native' data type is returned. Here is an example:

Note that when you call wrap(object) without the second parameter, Cyclos will try to determine which are all the available custom fields. You can also pass in the second argument, which is a collection of org.cyclos.entities.system.CustomFields to specify exactly which are the fields to use.

Also, note that you don’t need to wrap every object (we’ve seen many clients doing this). You only should wrap objects when reading / writing custom values.

Sometimes it might be interesting to have system-wide locks, specially in a cluster. Cyclos implements locks using PostgreSQL’s advisory locks. All locks are held until the transaction ends (either committed or rolled back). This means that locks cannot be manually released, so watch out for large transactions which can cause retention on other processes / requests because of locks.

Cyclos provides the org.cyclos.impl.locks.LockHandler component to acquire locks. Starting with version 4.16, custom locks are supported. These locks are meant to be used by scripts.

For an example in a custom lock usage, refer to this example.

From Cyclos 4.12 onwards, custom alerts can be defined for both system and users, and generated using scripts.

Administrators can be notified of these alerts. In the administrator’s notification settings page, there are options under User alerts and System alerts.

From Cyclos 4.16.14 onwards, custom notifications can be sent to users. They can be sent by the following mediums: email, sms messages and mobile app notifications. The mobile app notification uses Firebase Cloud Messaging (FCM) to send the notifications. How to configure FCM is included in the documentation of the mobile application. If you do not have access to it please contact STRO at info@cyclos.org.

However, distinct from regular notifications, users cannot opt out of custom notifications, so the recommended use case is to notify users of important events rather than simple / frequent notifications.

Cyclos provides the notificationHandler.custom() method, which returns a CustomNotificationSender. It works in a builder-style, providing the following methods:

After preparing all notification parameters, the notification must be sent using one of the following methods:

For an example for custom notifications, refer to this example.

Libraries are scripts which are included by other scripts, in order to reuse code, and are never used directly by other functionality in Cyclos.

Each script (including other libraries) can have any number of libraries as dependencies. However, circular dependencies between libraries (for example, A depends on B, which depends on C, which depends on A) are forbidden (validated when saving a library).

The order in which the code on libraries is included in the final code respects the dependencies, but doesn’t guarantee ordering between libraries in the same level. For example, if there are both C and B libraries which depend on A, it is guaranteed that A is included before B and C, but either B or C could be included right after A. So, in the example, your code shouldn’t rely on that B comes before C. In this case, the library C should depend on B to force the A, B, C order.

Contrary to other script types, libraries don’t have bound variables per se: the bindings will be the same as the script including the library.

Also, as libraries are just prepended in other scripts, no direct examples are provided here. The example scripting solutions, however, all use libraries.

These scripts are used to perform custom validation logic to custom field values. The field can be of any type (users, advertisements, user records, transactions and so on).

These scripts are used to load a list of allowed values for a custom field. Custom fields of type dynamic selection are required to have such a script. Several other field types can have an optional load values script: string, integer, decimal, date, url, enumerated or linked entity. Enumerated fields naturally have a list of static possible values. The script, however, can be used to show a subset of those options to specific users. Multi-line text, rich text, boolean, image and file types cannot have a load custom field values script.

If a custom field of type string, integer, decimal, date, url or linked entity has a load values script, Cyclos will use a single selection or radio button group widget instead of the regular widget for the custom field. Also, when a load custom field values script is used, the server-side validation will ensure that saved values are valid according to the allowed values list.

The script has a separated code block which loads values for custom fields being used as a search filter. The field types supporting load values when filtering are: dynamic selection, linked entity and enumerated. In that case, the bound variables will be different from the ones for the code block that runs over fields used to create or edit some entity (user, advertisement, record, etc.).

This kind of script is responsible for generating account numbers, in case more control than the default (random generation) is needed.

These scripts are used to calculate the amount of an account fee over a specific user. An account fee is charged periodically or manually over many accounts, according to the Charged account fees setting in member products.

These scripts are used to either calculate the amount or the full characteristics of a transfer fee (a fee triggered by another transfer).

Starting with Cyclos 4.16, the transfer fee details page has a field called 'Configure fee via', which can be set to either 'Form' or 'Custom script'.

These scripts are used to determine to which status(es) a transfer may be set after the current status. By default, if no script is used, the possible next statuses (as configured in the transfer status details page) will be available.

Using a script, however, allows using finer-grained controls. For example, a specific status could be allowed only by specific administrators, or only under special conditions (for example, checking the account balance or any other condition).

These scripts can be used to manage user sessions (logins) externally. It can only be set in the network default configuration, as the custom session handling script. There are distinct code blocks in this script type, each implementing a different aspect.

On any of these functions, returning null or having an empty code block will result in the default session management taking place. This way, it is possible to implement a custom handling only on special cases. For example, a custom session mechanism might be used only for privileged administrators, whose session tokens comply with a specific format.

For reference, Cyclos sessions use 32-character alphanumeric strings, with no punctuation. So, for example, if session tokens generated for those administrators have a different format, say, a UUID, the script can differentiate which sessions tokens correspond to normal sessions (and return null on the Logout and Resolve functions for those token format) and handle only those specific sessions. Also, in such a case, the login method could check the user group being logged in, and either perform the login on the underlying system (returning the generated session token) or return null for regular users.

Caution: Errors on any of these functions, specially the first three, may cause users not being able to login or access the system. A good security measure while developing such scripts is to handle a specific (for example, if the login name is 'admin') with the default session resolution, and withdraw this case after the rest of the script is ready. If such a situation occurs, a possible workaround is to login in global mode, then disable and lock the custom session handling in the configuration from which the network configuration inherits. Then edit the script and unlock it again in global mode.