Automate Your Sync the Smart Way with Rapidi Scheduling
When you have data flowing between two systems, getting the timing right is just as important as getting the data mapping right. In Rapidi, scheduling controls when and how your transfers run — including how often, in what order, and what happens when runs overlap.
This article covers how to structure schedules for data synchronization, including choosing run frequencies, preventing overlaps, and designing dependencies so transfers execute in the proper sequence.
A Quick Summary
Schedule structure: How schedules, groups, and transfers interconnect to form your sync automation.
Run windows: Selecting appropriate run frequencies for different data types — real-time vs. batch.
Overlap prevention: How to prevent consecutive runs from interfering with each other.
Dependencies: Managing sequential vs. parallel execution so transfers run in the right order.
Monitoring: Tracking schedule performance and spotting issues before they affect your data.
Watch the Full Session
This article is based on Session 10 of Rapidi's Open Office Hours training program. The full session includes a live walkthrough of scheduling in MyRapidi, a step-by-step setup demo, and practical examples of how to design reliable sync schedules.
What Is Scheduling in Rapidi?
Scheduling in Rapidi is the mechanism that automates when your transfers run. Instead of manually triggering each transfer, you define schedules that run transfers automatically at set intervals — every five minutes, every hour, once a day, or on any custom frequency.
A schedule brings together one or more transfer groups, each containing transfers that need to run together. This structure lets you organize your sync logic clearly: which transfers run, when they run, and in what order.
Without a well-designed schedule, transfers may run at the wrong time, overlap with each other, or execute in the wrong order — leading to data inconsistencies, missed records, or unnecessary API calls.
How Schedules, Groups, and Transfers Connect
The scheduling hierarchy in Rapidi works like this:
-
Schedule: The top-level container. It defines the run frequency (e.g., every 10 minutes) and holds one or more groups.
-
Group: A collection of transfers within a schedule. Groups can run sequentially (one after another) or in parallel (all at once).
-
Transfer: The actual data sync operation. Each transfer moves data from a source to a destination for a specific object type (e.g., customers, invoices, products).
This three-level structure gives you fine-grained control over execution order and timing.
Choosing the Right Run Frequency
Not all data needs to sync at the same frequency. The right interval depends on how time-sensitive the data is and how much volume you are processing:
- Real-time or near-real-time (1–5 minutes): Best for critical data like new orders, price changes, or inventory updates that affect customer-facing operations.
- Frequent (10–30 minutes): Suitable for most CRM-to-ERP syncs — customer updates, contact changes, opportunity stages.
- Hourly or daily: Appropriate for reference data, historical records, or batch processes where immediacy is not required.
Running everything at the highest frequency is tempting but wasteful. It consumes API calls and can create unnecessary load on your systems. Match the frequency to the business need.
Preventing Overlaps Between Runs
One of the most common scheduling pitfalls is overlap — when a new run starts before the previous one has finished. This can happen when transfer volumes are large or when API response times are slow.
Rapidi provides built-in overlap prevention. When a schedule is configured to prevent overlaps, a new run will not start if the previous run is still in progress. This ensures data integrity and prevents duplicate processing.
Best practice: Always enable overlap prevention for schedules that process high volumes or have variable execution times. If a run consistently overlaps, consider increasing the interval or splitting the workload across multiple schedules.
Managing Dependencies: Sequential vs. Parallel
Transfer dependencies determine the order in which your syncs execute. Getting this right is critical for data integrity:
- Sequential execution: Transfers run one after another. Use this when Transfer B depends on data created by Transfer A. For example, sync customers before syncing their invoices — because the invoice transfer needs the customer record to already exist in the destination.
- Parallel execution: Transfers run simultaneously. Use this when transfers are independent of each other. For example, syncing products and contacts at the same time — neither depends on the other.
In Rapidi, you control this through groups. Transfers within a group can be set to run in sequence or in parallel. Groups themselves can also be ordered sequentially within a schedule.
Designing Your Schedule Structure
A well-designed schedule structure reflects your data dependencies and business priorities. Here is a common pattern:
-
Group 1 — Master data (sequential): Sync customers, vendors, and products first. These are the records that other transfers depend on.
-
Group 2 — Transactional data (sequential): Sync orders, invoices, and shipments. These reference the master data records created in Group 1.
-
Group 3 — Reference data (parallel): Sync categories, units of measure, and other lookup data. These are independent and can run simultaneously.
This pattern ensures that dependent data is always available before it is needed, while independent data syncs as fast as possible.
Monitoring Schedule Performance
Once your schedules are running, monitoring is essential. Rapidi provides schedule logs that show:
- Run start and end times: How long each run takes
- Records processed: How many records were synced in each transfer
- Errors and warnings: Any issues that occurred during the run
- Overlap occurrences: Whether any runs were skipped due to overlap prevention
Regularly reviewing these logs helps you spot trends — like runs that are gradually taking longer, which may indicate growing data volumes or API throttling.
Common Scheduling Mistakes to Avoid
- Running everything at the same frequency: Match frequency to business need, not convenience.
- Ignoring dependencies: If Transfer B needs data from Transfer A, they must run in sequence — not in parallel.
- No overlap prevention: Without it, long-running transfers can pile up and cause data issues.
- Too many transfers in one schedule: If a schedule becomes too large, split it into focused schedules with clear responsibilities.
Watch all sessions and register for upcoming ones: rapidionline.com/resources/open-office-hours
See the full Season 1 schedule: rapidionline.com/product-updates/open-office-hours-season-1
Frequently Asked Questions
How often should I run my schedules?
It depends on the data type. Critical operational data like orders or inventory should sync every 1–5 minutes. Standard CRM data like contacts and opportunities typically works well at 10–30 minute intervals. Reference data and historical records can sync hourly or daily. The key is matching the frequency to how time-sensitive the data is for your business processes.
What happens if a scheduled run overlaps with a previous run?
If overlap prevention is enabled, the new run will be skipped and the previous run will continue to completion. This prevents duplicate processing and data integrity issues. If overlap prevention is not enabled, both runs will execute simultaneously, which can lead to conflicts, duplicate records, or unexpected results. Always enable overlap prevention for production schedules.
Can I run some transfers in parallel and others in sequence within the same schedule?
Yes. Use groups within your schedule to control execution order. Transfers within a group can run in parallel if they are independent. Groups themselves run in sequence, so you can ensure that master data syncs (Group 1) complete before transactional data syncs (Group 2) begin. This gives you both speed and proper ordering.
How do I know if my schedule interval is too short?
Check your schedule logs for overlap occurrences. If runs are frequently being skipped because the previous run has not finished, your interval is too short for the volume of data being processed. Either increase the interval, optimize the transfers (e.g., use filters to reduce the record set), or split the workload across multiple schedules.
Should I create one large schedule or multiple smaller ones?
Multiple smaller schedules are generally better. They are easier to monitor, debug, and maintain. Group related transfers together — for example, one schedule for customer data, another for financial data, and another for product data. This also lets you set different frequencies for each group based on business priority.
