Move data between systems without scripts
Most data movement does not need a warehouse or a pipeline framework. It needs a scheduled job that reads one system, reshapes a few columns, and writes another — and, critically, that somebody other than its author can still understand in six months.
How to set it up
- 1
Read narrowly
A query with a WHERE clause beats reading a whole table and filtering afterwards. The database is much better at this than the pipeline is.
- 2
Cast on the way in
Especially from a spreadsheet, where everything arrives as text and stays that way unless you say otherwise.
- 3
Choose the write mode deliberately
Append accumulates, replace truncates first, upsert merges on a key. All three are right sometimes, and picking by accident is how tables end up wrong.
- 4
Schedule to match the source
A table that changes hourly does not need a five-minute job; one that changes weekly does not need a daily one.
The connectors this uses
Questions
Is this a replacement for Fivetran or Airbyte?
No, and it would be dishonest to claim so. Those move large volumes into warehouses for data teams. This moves modest volumes between the tools a small team already uses, with no warehouse involved.
Can I transform the data in between?
Within limits, on purpose. Column selection, row filters, type casts and computed fields using small expressions — abs, round, min, max, len. Not a scripting environment, so business logic cannot hide somewhere nobody thinks to look.