← All guides
dbSliceSynchronization / Mapping / Masking

Move the Right Data. Mask It Before It Arrives.

How dbSlice combines selective synchronization, column mapping, and in-flight masking to prepare useful data for development, testing, integration, and local read models.

Technical product article · English edition · October 2026

A staging environment needs realistic relationships and representative records. A branch application needs a current catalog. An integration needs selected fields in a different structure. None of these requirements automatically calls for a complete, unchanged copy of a production database.

dbSlice brings these decisions into an explicit workflow: choose a source, select the records, map the destination columns, configure masking, and execute a transfer you can inspect and repeat. The result is a practical way to move useful data while controlling which values reach the destination.

The Windows product provides a graphical interface and a CLI, with support for SQL Server, PostgreSQL, Oracle, MySQL, MariaDB, Firebird, SQLite, and Caché/IRIS. Its public product page describes filtering, mapping, masking, reconciliation, saved jobs, and operational recovery features. dbSlice product overview

1. Three decisions belong in every transfer

Before moving a row, answer three questions: Which records are needed? Which columns belong in the target? Which values must change before being written?

01 · SelectTable, view, or query
Define the row subset
02 · MapMatch destination columns
Preserve the matching key
03 · MaskTransform selected values
Before the target write
04 · DeliverDatabase or export file
Validate the outcome

The synchronization path maps and transforms values before writing destination batches. File export is a separate operation with its own format and masking options.

Selection reduces the scope of the operation. Mapping makes differences between schemas explicit. Masking changes the values that a downstream environment receives. Together, these controls support tasks such as refreshing a test environment, preparing a migration, feeding an external process, or maintaining a local lookup database.

For example, a service catalog may need descriptions and categories at a branch location, while a support test environment may need customer relationships without usable customer email addresses. These are different datasets and should be configured as different operations.

2. Select a useful slice and preserve its relationships

In Synchronize, choose the source connection and a table, view, or SELECT query. Choose a destination connection, target object, and matching key. Apply a source-side WHERE condition when the destination needs only a subset.

A synthetic service catalog could use:

category_id = 1

The selection also creates a dependency question: do the required category and type records exist at the destination? Transfer parent records before dependent records, and validate the references after the load. A filtered transfer does not, by itself, define a complete dependency-aware slice of every related table.

Use Map columns when names differ or the target needs a different projection. A mapping can connect full_name to display_name, for example, while retaining id as the matching key. Review lengths, nullability, destination defaults, and data types before running the operation. Omitting a required target column is only viable when the target can supply a valid value.

When minimizing the data read from the source is itself a requirement, use a view or SELECT that returns only the necessary columns, including a suitable key. Excluding a column from the target mapping should not be treated as proof that it was never fetched from the source.

3. Mask values before the destination write

The important placement of masking is inside the transfer process. In the reviewed synchronization implementation, source rows are read, destination rows are constructed through the mappings, selected values are transformed, and the resulting batch is written to the destination.

This avoids a workflow that first loads original values into a test table and only masks them afterward. Correctly configured masking changes the relevant values before that write occurs.

Synthetic sourceid: 42
name: Alice Example
email: [email protected]
Configured rulesid: preserved key
name: template:Customer {key}
email: template:user-{key}@example.test
Destination valuesid: 42
name: Customer 42
email: [email protected]

Illustrative values only. The key remains available for matching records; the name and email are replaced using explicit mapping rules.

In the mapping dialog, enable obfuscation for the relevant non-key columns and provide a rule where a default mask is unsuitable. The reviewed implementation supports these examples:

Rule Synthetic input Result Design consideration
fixed:REDACTED Alice Example REDACTED All rows receive the same replacement; check uniqueness constraints.
template:Customer {key} Alice Example, key 42 Customer 42 Produces a readable label while exposing the retained key.
template:user-{key}@example.test [email protected], key 42 [email protected] Replaces the original mailbox and domain with a test value.
keep:2,* ABCDE AB*** Keeps the first two characters, not the last two.
email: [email protected] al***@example.test Retains a prefix and the original domain.
cpf: 123.456.789-00 ***.***.***-** Replaces digits and preserves punctuation; does not generate a valid replacement identifier.

These outputs were checked against the masking function using synthetic values. They are not results from copying a live customer table.

With no explicit rule, masking is type-based. Examples in the reviewed implementation include zero for numeric values, false for Booleans, a fixed historical date for dates, and a short masked string for text. These replacements can change business meaning, collapse unique values, or conflict with destination validation. Check the output and use explicit rules where needed. An unrecognized rule falls back to the default mask, so validate the rule spelling as well as the resulting values.

4. Preserve keys deliberately

Keys are necessary for matching records and maintaining relationships. The synchronization mapping interface protects the selected key and recognized destination foreign keys from masking. Export and job-level obfuscation resolve a broader default scope that excludes the selected key and recognized primary and foreign keys.

That behavior helps keep a usable dataset, but retained identifiers still deserve review. A customer ID, natural key, or foreign-key chain may link a row to other information. Query results can also lack the foreign-key metadata available for a table.

Do not assume that checking an obfuscation option means every sensitive field has been discovered and removed. Review free text, secondary identifiers, unmapped relationships, and any values retained by partial masks. If the intended dataset requires consistent replacement of identifiers throughout a relationship graph, design that transformation explicitly rather than changing a key in isolation.

5. What “in flight” means for protection

For a direct database-to-database synchronization, the reviewed service moves batches through memory and does not require a CSV or JSON staging dump as an intermediate handoff. That can reduce the number of data-bearing files involved in a refresh.

It is a narrower statement than saying that no intermediate data or files can ever exist. The application must read original values before masking them. Database logs, host memory, diagnostic captures, and operational logs have their own protection requirements. An Export operation intentionally creates an output file.

Masking also differs from transport encryption and encryption at rest. Configure connection security for the database connection, protect credentials, restrict access to generated files, and apply the retention policy appropriate for the environment. Those controls serve different purposes.

Avoid describing a masked output as automatically anonymous or automatically compliant with GDPR or LGPD. The EDPB distinguishes pseudonymisation, which reduces linkability, from anonymisation, which removes the link to an individual. A masking rule alone does not settle that distinction for a dataset. EDPB: anonymisation and pseudonymisation

The product's practical contribution is configurable transformation before delivery. Compliance depends on the dataset, the remaining information, the recipients, and the surrounding organizational and technical controls.

6. Choose the synchronization behavior

The selected mode determines the effect on the destination. Make that effect part of the operation's definition, alongside filtering and masking.

Mode Intended use Point to review
Initial load / insert only Populate a prepared target Existing keys and constraints may affect insertion.
Insert new Add records whose keys are not present Existing records are not refreshed.
Upsert Insert missing keys and update matching records Rows absent from the selected source are not automatically deleted.
Update existing only Refresh records already present Missing target records are not inserted.
Full reload Replace the target contents for a new load Review the destructive effect and the recovery plan before execution.

The deletion point matters especially for filtered data. If an item moves out of the chosen category, it can disappear from the next source selection while remaining in an Upsert target. Define how obsolete or out-of-scope records will be handled.

Batch processing makes large transfers manageable, and the reviewed synchronization pipeline can overlap reading with writing. This is not a promise of unlimited throughput or a single transaction spanning the entire dataset. Choose a page size suitable for row width, destination write behavior, network conditions, and available memory.

7. Recovery: retries and checkpoints have a defined scope

The current synchronization service includes retries for exceptions classified as transient. In the reviewed code, a retryable operation has up to three attempts, with delays of 500 ms and 1 second before the later attempts. Ordered page reads and destination batch writes use this mechanism.

This handles some temporary interruptions without requiring the operator to restart immediately. It does not resolve invalid data, missing permissions, persistent connectivity failures, or every possible database error. After the attempts are exhausted, the failure still needs attention.

Checkpoint support records progress after completed batches for eligible operations. The reviewed service excludes unordered streaming and composite tie-break pagination from this checkpoint path; those configurations should not be advertised as having the same resume behavior. Confirm the recovery options exposed by the installed build and test them with the selected provider and mode.

A completed batch and its checkpoint are also different events. Recovery design should account for an interrupted write acknowledgement or checkpoint save. Review the target state before rerunning a failed load, particularly when insertion, triggers, or other side effects are involved.

The useful promise is bounded retry handling and supported progress recovery. It is not automatic recovery from every failure or a guarantee of exactly-once delivery across an entire distributed system.

8. From one destination to local SQLite read models

The same controlled-transfer workflow can prepare a local SQLite catalog for an application or branch. Each destination needs an explicit scope, refresh schedule, validation process, and freshness policy.

For example, a catalog may tolerate a scheduled refresh while a transaction requiring current inventory must still use the authoritative system. During a connection outage, the local copy contains the last successfully delivered state. The application determines which functions can use it and when that state is too old.

The reviewed transfer path reads tables, views, or queries. Do not infer native transaction-log CDC from an illustration of a synchronization engine. Scheduling repeated transfers, or filtering on a timestamp maintained by the application, is different from capturing every committed database change. Incremental selection needs its own rules for updates, late arrivals, and deletions.

Similarly, a source connected to many SQLite icons is a deployment concept, not evidence of a built-in fleet control plane. Distribution to many nodes needs endpoint management, scheduling, concurrency limits, monitoring, licensing appropriate to deployment, and application-level consistency decisions. Start with one validated destination before increasing the number of consumers.

9. Build a repeatable workflow through GUI and CLI

For a first masked synchronization, use the graphical interface to make the operation reviewable:

  1. Choose the source and target connections and confirm the intended environments.
  2. Select the source object, destination object, and matching keys.
  3. Apply a small, representative filter for the initial validation.
  4. Open Map columns, review included fields, and configure masking for selected non-key columns.
  5. Choose the synchronization mode and page size with the destination effect in mind.
  6. Execute the operation and inspect the completion status, warnings, and destination samples.
  7. Validate references, expected replacements, required columns, and any unique values.
  8. Save the configuration and review the job representation before scheduling a recurring run.

GUI mapping rules and a job's broad obfuscation flag should not be assumed to express the same policy. Confirm that a saved or exported configuration carries the column mappings and rules needed by that specific workflow. Inspect the produced job rather than manually inventing unsupported JSON properties.

The CLI allows suitable saved jobs to run without the interactive interface. Use the application's command-generation facilities for the installed version, then configure the external scheduler's identity, access, failure handling, and overlap policy.

Linux services, Docker deployment, Kafka integration, and bespoke scheduling pipelines shown as consulting options in the supplied material should be scoped as separate integration work. They should not be presented as components automatically included in the Windows desktop application.

10. Validate the transformed result, not only the row count

A matching row count is useful, but it cannot prove that the right data arrived or that masking was applied correctly.

Check What it establishes
Keys and selected scope The expected records reached the destination.
Parent-child references The slice remains usable across dependent tables.
Mapped values and types Fields reached the intended columns with valid representations.
Masked samples Explicit rules produced the expected replacements.
Unique and required fields Replacements did not break application constraints.
Out-of-scope target rows A refresh did not leave an unmanaged residue.
Completion and recovery records The operation succeeded or has an understood recovery state.

For a masked column, compare the destination with the expected transformed value, not with the original source value. A raw value comparison should detect a difference there. Separate those intentional differences from missing records and unexpected mapping changes.

In the supplied earlier demonstration, a SQL Server-to-PostgreSQL Upsert processed 245,014 service records in 10.6 seconds. A saved-job execution reported 10.7 seconds. Those observations illustrate a particular transfer workload. They do not establish masking overhead, CDC latency, SQLite fleet performance, or an end-to-end security guarantee. The combined upsert/update counter also does not independently prove how many previously stored values actually changed.

11. Start with a controlled refresh

A useful first deployment is a small, repeatable refresh whose purpose is clear: a representative test slice, an integration table, or a local catalog. Define the subset, map the columns, specify masking rules, verify the target, and record how the operation will be repeated and recovered.

That is the central value of dbSlice in this workflow: making data movement configurable and inspectable, while placing transformation before destination writes. Expand the volume, schedule, and number of consumers after the behavior has been validated for the intended environment.

Explore the product and installation options at dbslice.net.

Sources and scope

This article develops the synchronization, masking, and delivery themes in the supplied illustrations. Its product-specific details were checked against the accompanying implementation: SyncService, ColumnMappingForm, ObfuscationScope, and ObfuscationMask. Retry counts, checkpoint conditions, and masking rules are implementation observations and may change between releases.

External references: dbSlice product overview and EDPB terminology on anonymisation and pseudonymisation. The diagrams in this article are explanatory layouts, not a claim that a multi-node deployment was executed. All masking examples use synthetic values. No live database transfer was performed to create this article.