Historical Data Migration Guide

When you connect a new platform to FluxCascade, you likely have existing data in both systems. This guide walks you through migrating historical data safely — without creating duplicates or losing information.

Overview

Goal: Populate both systems with each other's existing records so they start from a consistent baseline before ongoing sync begins.

What You'll Achieve:

• All existing source records created in the target system
• Matching records linked (not duplicated)
• Clean data foundation for ongoing sync

Time Required: 30-60 minutes setup, plus sync runtime (depends on record count)

Before You Start

Assess Your Data

Before running a migration, answer these questions:

1. How many records? This affects sync duration and API rate limits
2. How clean is the data? Duplicates and incomplete records cause problems
3. Which direction? Usually one system is more complete than the other
4. What's the matching key? Email, external ID, or another field

Record Count Guidelines

Step 1: Clean Your Source Data

Invest time in data quality before migrating. Fixing issues after records are synced to both systems is much harder.

Deduplicate

Check for duplicate records in your source system:

• Multiple contacts with the same email
• Company names with slight variations ("ABC Corp" vs "ABC Corporation")
• Phone numbers that match when formatted

Fill Required Fields

Ensure required fields have values:

• Every contact needs an email (if using email matching)
• Names should be populated (avoid blank first/last names)
• Phone numbers should be in a parseable format

Standardize Formats

Apply consistent formatting before migration:

• Capitalize names properly
• Standardize phone numbers
• Use consistent address formatting

Step 2: Set Up Your Mapping

Create the mapping as you normally would (see the HubSpot + Jobber guide for a full walkthrough), with these migration-specific settings:

Direction

For the initial migration, use one-way sync from your more complete system to the less complete one:

More complete system → Less complete system
(e.g., HubSpot → Jobber if HubSpot has more records)

After the migration completes, you can change to bidirectional.

Matching Strategy

Configure matching before the migration to prevent duplicates:

1. Set email as the primary matching field
2. Enable Case Insensitive Matching
3. Consider adding a secondary matching field (phone or name combination)

Records that match will be updated, not duplicated. Only unmatched records create new entries.

Step 3: Run a Test Sync

Before migrating all data, test with a small subset:

1. Open your mapping
2. Click Test Sync
3. Review the preview:
   • How many records will be created vs. updated?
   • Are field values transforming correctly?
   • Are there any errors?

What to Check

• Created count: Does this match the number of records you expect to add?
• Updated count: These are existing matches — verify a few look correct
• Skipped count: Records skipped due to missing required data
• Error count: Records that failed transformation or validation

If the test reveals issues, fix your mapping configuration and test again.

Step 4: Run the Full Migration

When the test looks good:

1. Click Sync Now
2. Select Full Sync (not incremental — you want all records)
3. Start the sync

Monitor Progress

While the migration runs:

1. Go to Syncs in the sidebar
2. Click on the active sync job
3. Watch real-time statistics:
   • Records processed / total
   • Created, updated, skipped, failed counts
   • Estimated time remaining

If Something Goes Wrong

You can pause a running sync at any time:

1. Click the Pause button on the active sync
2. Review the records processed so far
3. Fix any issues in your mapping
4. Resume to continue from where it left off, or Cancel to stop

Records already synced are not rolled back on pause or cancel — only future records are affected.

Step 5: Verify the Migration

After the sync completes:

Check Record Counts

Compare record counts between systems:

Source system:  1,250 contacts
Target system:  1,245 contacts (5 skipped due to missing email)
FluxCascade:    Created 800, Updated 445, Skipped 5

The numbers should add up. If they don't, check the sync error log.

Spot-Check Records

Manually verify 10-20 records across both systems:

• Do field values match?
• Are transforms applied correctly (phone formatting, address structure)?
• Are there any unexpected duplicates?

Review Error Logs

Go to the completed sync's detail view and review any errors:

• Missing required fields: Clean the data and re-sync those records
• Transformation failures: Adjust transforms or fix source data
• API errors: Rate limits or permission issues — retry after a delay

Step 6: Run the Reverse Migration (If Needed)

If the target system also has unique records that don't exist in the source:

1. Create a second mapping in the reverse direction
2. Run a Full Sync from target → source
3. This populates the source with any records only in the target

Or Switch to Bidirectional

After both sides are populated:

1. Edit your original mapping
2. Change direction to Bidirectional
3. Run one more Full Sync to reconcile any differences
4. Enable scheduled sync for ongoing synchronization

Step 7: Enable Ongoing Sync

Once migration is complete and verified:

1. Set sync frequency (every 15 minutes recommended for active data)
2. Optionally enable webhook-triggered sync for real-time updates
3. Configure email notifications for sync failures

Handling Large Migrations

For datasets over 5,000 records:

Schedule Off-Hours

Run the migration during low-activity periods to avoid:

• API rate limit conflicts with regular platform usage
• Confusion from records appearing in real-time for users of the target system

Rate Limit Awareness

Each platform has API rate limits:

FluxCascade automatically throttles requests to respect these limits, which is why large migrations take longer.

Troubleshooting

Duplicate Records Created

Cause: Matching field was empty or inconsistent

Solution:

1. Identify duplicates in the target system
2. Merge them in the target platform
3. Ensure matching fields are populated
4. Re-run sync — FluxCascade will match on subsequent syncs

Migration Timeout

Cause: Very large dataset exceeding session limits

Solution: Contact support for assisted migration with batch processing

Partial Migration (Some Records Missing)

Cause: Records skipped due to validation errors

Solution:

1. Check the sync error log for specific records
2. Fix the data in the source system
3. Run another Full Sync — only missing records will be created

Data Looks Different After Migration

Cause: Transforms changing the display format

Solution: This is expected for fields with transforms (e.g., phone numbers reformatted to E.164). Verify the data is correct, just formatted differently.

Best Practices

1. Always test first — Run a test sync before the full migration

2. Migrate in one direction first — Don't start with bidirectional; establish a baseline, then open the other direction

3. Document your migration — Note the date, record counts, and any decisions made

4. Clean data before, not after — Fixing data in one system is easier than fixing it in two

5. Keep a backup — Export your target system's data before the migration in case you need to reference the pre-migration state

6. Communicate with your team — Let users know when the migration will run and that new records will appear in their systems

Next Steps

• HubSpot + Jobber Guide — Set up your first integration
• Error Handling — Handle sync failures during and after migration
• Scheduling — Configure ongoing sync after migration