CredVault
Dashboard

Migrations

Migrate data from your existing systems into CredVault safely and reliably.

What are Migrations?

Migrations let you:

  • Import data from other databases
  • Transfer between CredVault instances
  • One-time or continuous sync
  • Test before production
  • Rollback if needed

Getting Started

Plan Your Migration

Before starting:

  1. Identify source - Where is your data now?
  2. Choose destination - Which CredVault database?
  3. List data sources - All tables/collections to migrate
  4. Plan schedule - When to run?
  5. Test first - Use non-production environment

Create Migration

  1. Click Migrations in sidebar
  2. Click New Migration
  3. Name it: "Postgres to CredVault Migration"
  4. Choose type:
    • One-time - Copy data once
    • Continuous - Keep syncing
    • Scheduled - Run on schedule
  5. Next: Configure source

Configuring Source

Connection Details

Provide source database info:

Connection Type: PostgreSQL
Host: source-db.example.com
Port: 5432
Database: old_database
Username: migration_user
Password: [secure password]

Test connection before proceeding.

Select Tables

Choose which tables to migrate:

☑ customers (1.2M rows)
☑ orders (5.3M rows)
☑ products (25K rows)
☐ internal_logs (skip)
☐ archive_data_2020 (skip)

Preview Data

See sample of what will be migrated:

customers table:
customer_id | name            | email
------------|-----------------|-------------------
1           | Alice Smith     | alice@example.com
2           | Bob Johnson     | bob@example.com
3           | Carol White     | carol@example.com
... (showing 3 of 1,234,567 rows)

Data Mapping

Column Mapping

Tell CredVault how columns map:

Source Column → Destination Field
customer_id → id (rename)
email → user_email (rename)
created_date → created_at (rename)
[skip] → internal_tag (skip column)

Type Conversion

Automatic conversion:

Source Type → Target Type
VARCHAR → String
INTEGER → Integer
DECIMAL → Float
TIMESTAMP → Datetime
BOOLEAN → Boolean

Override if needed:

Source: customer_tier (string: "gold", "silver")
Target: tier_code (integer: 1, 2)
Conversion: Create mapping "gold"→1, "silver"→2

Data Transformation

Transform data during migration:

# Uppercase all email addresses
email = email.upper()

# Convert date format
created_at = parse_date(created_at, format='MM/DD/YYYY')

# Trim whitespace
name = name.strip()

# Calculate new field
full_age = current_year - birth_year

Migration Process

Pre-Migration Checks

System verifies:

  • ✓ Source is accessible
  • ✓ Destination has space
  • ✓ Column mappings valid
  • ✓ No data conflicts
  • ✓ Permissions correct

Running Migration

  1. Click Start Migration
  2. System begins:
    • Reading from source
    • Transforming data
    • Writing to destination
    • Tracking progress
  3. See real-time status:
Progress: ████████░░ 82%
Migrated: 1,024,567 / 1,234,567 rows
Time elapsed: 23 minutes
Estimated: 5 minutes remaining
Speed: 45,000 rows/minute

Monitoring

During migration, monitor:

  • Progress percentage
  • Rows processed
  • Errors encountered
  • Network/CPU usage
  • Current step

Handling Errors

If errors occur:

Error on row 45,632:
Column 'customer_id' has invalid value: "ABC123"
Expected: Integer

Options:
○ Skip row (continue)
● Fail and stop
○ Mark for review (continue, flag later)

Choose how to handle:

  • Skip - Continue without this row
  • Stop - Halt migration
  • Flag - Continue but mark for manual review

Validating Data

Post-Migration Checks

After migration completes:

Source Database:
- customers: 1,234,567 rows
- orders: 5,234,891 rows
- products: 25,643 rows
Total: 6,495,101 rows

CredVault Database:
- customers: 1,234,567 rows ✓
- orders: 5,234,891 rows ✓
- products: 25,643 rows ✓
Total: 6,495,101 rows ✓

Validation: PASSED

Data Reconciliation

Compare source and destination:

Record count match: ✓
Column count match: ✓
Data types match: ✓
No null mismatches: ✓
Checksum: ✓

Sample Verification

Spot-check random samples:

Checking 100 random rows:
Source | Destination | Match
456    | 456         | ✓
8923   | 8923        | ✓
124567 | 124567      | ✓
... (100 of 100 match)

Verification: PASSED

Continuous Migration

Setup Sync

For continuous sync:

  1. Set sync frequency:
    • Every 5 minutes
    • Hourly
    • Daily
    • Custom interval
  2. Choose sync mode:
    • Full Sync - Recopy everything
    • Incremental - Only new/changed rows
    • CDC - Change Data Capture

Monitoring Sync

Last sync: 2024-06-12 14:30:22 UTC
Status: Success
Duration: 2 minutes
New rows: 143
Updated rows: 87
Deleted rows: 12
Next sync: 2024-06-12 14:35:22 UTC

Pause/Resume

Pause sync during:

  • Maintenance windows
  • Source system updates
  • Destination backups

Resume when ready.

Rollback and Recovery

Before Migration

Create backup:

CredVault automatically creates backup
Backup name: migration_2024_06_12_backup
Retention: 30 days
Can restore anytime

After Problems

Rollback migration:

  1. Click Rollback
  2. Choose restore point
  3. Restore to previous state
  4. Verify data restored
  5. Debug issue

Testing First

Test in staging environment:

1. Create staging CredVault instance
2. Run migration to staging
3. Validate data
4. Check application works
5. Then run to production

Best Practices

Plan Thoroughly

  • Map all columns before starting
  • Test transformations
  • Identify constraints
  • Plan for downtime

Test First

  • Run on non-production data
  • Verify output
  • Test application with migrated data
  • Practice rollback

Monitor Closely

  • Watch progress
  • Check error logs
  • Validate after completion
  • Keep audit trail

Communicate

  • Notify users of maintenance
  • Provide timeline
  • Update on progress
  • Confirm completion

Common Workflows

One-Time Database Migration

1. Set up connection to old database
2. Select all tables
3. Map columns
4. Test in staging
5. Schedule migration
6. Backup destination
7. Run migration
8. Validate
9. Switch applications over

Regular Data Import

1. Create CSV/API connection
2. Select columns to import
3. Map to CredVault fields
4. Test with sample
5. Schedule weekly import
6. Monitor for errors
7. Review new data

Multi-Source Consolidation

1. Identify data sources (3+ databases)
2. Design unified schema
3. Create migration for each source
4. Map overlapping fields
5. Handle deduplication
6. Run all migrations
7. Validate consistency

Troubleshooting

Connection fails

  • Check connection string
  • Verify firewall rules
  • Test credentials
  • Check network connectivity

Data mismatch

  • Verify column mapping
  • Check data transformations
  • Look for encoding issues
  • Compare sample rows

Too slow

  • Increase batch size
  • Use incremental mode
  • Reduce concurrent operations
  • Check network bandwidth

Storage exceeded

  • Clean up unused data
  • Archive old records
  • Reduce retention period
  • Upgrade storage plan