Performance Optimization Guide

As your data grows, sync performance becomes important. This guide covers practical techniques to keep your FluxCascade syncs fast and efficient.

Understanding Sync Performance

Every sync involves three time-consuming operations:

1. Data fetch — Reading records from the source platform's API
2. Transformation — Processing field values through transforms
3. Data write — Creating or updating records in the target platform's API

Most of the time is spent on API calls (steps 1 and 3). Optimization focuses on reducing the number and size of those calls.

Strategy 1: Use Incremental Syncs

Full syncs re-process every record, even those that haven't changed. Incremental syncs only process records modified since the last sync.

How to Enable

1. Open your mapping Settings
2. Set Sync Mode to Incremental
3. Save

Impact

For most ongoing syncs, incremental mode reduces processing by 90%+.

When to Use Full Sync

Keep full sync for:

• Initial data migrations (see Data Migration guide)
• Periodic reconciliation (e.g., weekly full sync to catch anything missed)
• After changing mapping configuration

Strategy 2: Enable Webhook-Triggered Syncs

Instead of polling on a schedule, webhooks let the source platform tell FluxCascade exactly when a record changes.

Benefits

• Near-instant sync — Changes propagate in seconds, not minutes
• Minimal API usage — Only changed records are fetched
• No wasted cycles — No polling when nothing has changed

How to Enable

1. Open your mapping Settings
2. Enable Real-time Sync
3. FluxCascade registers webhooks with the source platform automatically

See Webhooks for detailed setup and troubleshooting.

Combine with Scheduled Sync

The most reliable approach uses both:

• Webhooks for real-time updates (catches most changes instantly)
• Scheduled incremental sync every few hours as a safety net (catches anything webhooks missed)

Strategy 3: Optimize Field Selection

Every field in your mapping adds processing time. Map only what you need.

Audit Your Field Pairs

Review each field pair and ask:

• Is this field actively used in the target system?
• Would anyone notice if this field stopped syncing?
• Is this field changing frequently or is it static after creation?

Remove Unnecessary Fields

If a field is:

• Never viewed in the target system → remove it
• Only needed at creation time → set direction to source-only
• Rarely changing → consider a less frequent sync for that mapping

Impact

Each field pair adds:

• One read operation per record (source)
• One transform operation per record (if transform configured)
• One write operation per record (target)

Reducing from 20 field pairs to 10 roughly halves per-record processing time.

Strategy 4: Manage Rate Limits

API rate limits are the primary bottleneck for large syncs. FluxCascade handles throttling automatically, but you can help:

Stagger Sync Schedules

If you have multiple mappings, avoid running them simultaneously:

Mapping A (Contacts):  Every 15 min, starting :00
Mapping B (Deals):     Every 15 min, starting :05
Mapping C (Companies): Every 15 min, starting :10

This spreads API usage across the rate limit window.

Reduce Sync Frequency for Stable Data

Not all data changes at the same rate:

Monitor Rate Limit Usage

Check your sync logs for rate limit warnings:

Warning: Rate limit approaching for HubSpot API (85/100 requests used)
Throttling: Pausing 2 seconds before next batch

If you see frequent throttling, spread your syncs out or reduce sync frequency.

Strategy 5: Simplify Transforms

Complex transform chains add processing overhead per record. For large datasets, keep transforms lean:

Fast Transforms

These are essentially instant:

• trim, uppercase, lowercase, titlecase
• default_value
• Direct field mapping (no transform)

Slower Transforms

These require parsing or lookups:

• phone_e164 (parses and validates phone numbers)
• address_parse_* (parses address strings)
• value_map with many entries

For high-volume mappings, consider pre-cleaning data in the source system so fewer transforms are needed at sync time.

Monitoring Performance

Sync Duration Trends

Track how long your syncs take over time:

1. Go to Syncs
2. Look at the Duration column for recent sync jobs
3. If duration is increasing while record counts stay similar, investigate

Common Performance Degradation Causes

• Data growth: More records = longer syncs → switch to incremental
• New field pairs added: More fields = more processing → audit field necessity
• Transform changes: Complex new transforms → consider pre-processing
• Platform API slowdowns: External issue → check platform status pages

Setting Up Alerts

Configure email notifications for:

• Sync duration exceeding a threshold
• Sync failure alerts
• Rate limit warnings

Performance Checklist

Use this checklist when optimizing an existing mapping:

• [ ] Sync mode set to Incremental (not Full) for ongoing syncs
• [ ] Webhooks enabled for real-time data
• [ ] Only necessary fields are mapped
• [ ] Sync frequency matches data change rate
• [ ] Multiple mappings are staggered, not simultaneous
• [ ] Transforms are as simple as possible
• [ ] Periodic full sync scheduled (weekly) for reconciliation

Next Steps

• Scheduling — Configure sync frequency and timing
• Webhooks — Enable real-time sync
• Error Handling — Handle failures efficiently
• How Syncs Work — Understand the full sync lifecycle