Connect File URL / CSV to PostgreSQL
Read a CSV, TSV, JSON, JSON Lines or Excel file straight from a URL (or your stored files). Handles gzip/zip and picks the format automatically. Great for vendor exports dropped at a link.
What you can pull from File URL / CSV
A File URL / CSV job pulls records into a flat table on your schedule. Nested fields are flattened into ordinary columns, so what lands in PostgreSQL is ready to sort, filter and total rather than needing another cleanup step.
- CSV / TSV / JSON / Excel
- Auto format & compression
- Public URL or stored file
Options you can set
- location
- 'http' for a URL, 'storage' for a file in your account — defaults to
http - file_format
- auto, csv, tsv, delimited, json, jsonl or excel — defaults to
auto - compression
- auto, none, gzip, zip, bz2 or xz — defaults to
auto - has_header
- First row holds column names (delimited only) — defaults to
true - skip_rows
- Rows to skip before the header, for files with a preamble — defaults to
0 - encoding
- For files that are not UTF-8 — defaults to
utf-8 - max_rows
- Cap the rows a run reads
How it lands in PostgreSQL
PostgreSQL accepts append, replace, upsert writes. Upsert is the one that matters on a schedule: it merges on a key you choose, so matching rows are updated in place and only genuinely new records are created. Append instead, and a daily job multiplies your data.
Upsert uses key_columns as the conflict target.
- Rows are written in chunks of 1,000 by default, configurable up to 50,000.
- Replace truncates the target table before writing.
Authentication
No credential is needed for a public URL. If the file is not publicly reachable, upload it to your account storage and read it from there instead.
Keeping runs incremental
Not for this source: The whole file is re-read each run. Where that is expensive, narrow the job itself and run it less often.
Pair a narrowed read with an upsert write and a re-run costs almost nothing: the rows it already knows are updated, and nothing is duplicated. That combination is what makes a frequent schedule affordable.
A worked example
A nightly export, loaded and queryable
Read the export from its URL, cast the columns as they load, and upsert on the file’s own record id. Postgres is the right destination when the file needs joining against data you already hold.
Schedule: 30 2 * * * — read it back in plain English
Limits and pacing
These are the provider-side ceilings that shape a schedule, not ours. Knowing them up front is the difference between a job that runs quietly every morning and one that starts failing the week your data grows.
| Limit | Value | Applies to |
|---|---|---|
| File size Streamed rather than loaded whole, so the cap is about the pipeline. | 256 MB | File URL / CSV |
| Redirect hops The destination is re-validated at every hop. | 5 | File URL / CSV |
Common gotchas
Most of what goes wrong with a scheduled sync is not a bug — it is a detail of how one side behaves that nobody wrote down. These are the ones that come up for this pair.
Format detection trusts the extension first
Then the Content-Type header. Override it explicitly when a URL lies about what it serves, which happens more often than it should.
The link must return the current file
A URL that returns yesterday's export forever will sync perfectly and tell you nothing. This is the most common cause of a job that looks healthy but reports stale numbers.
Set either table or query, never both
The config rejects both being set, and rejects neither - caught when the job is saved rather than on its first run.
Set it up in four steps
- 1
Connect File URL / CSV
A public HTTPS link, or a file in your own account storage. No credential needed for a public URL.
- 2
Connect PostgreSQL
Host, port, database, user and password, stored encrypted.
- 3
Shape the data
Select the columns you want, filter rows, cast types and add computed fields. Everything else is dropped before it reaches the destination.
- 4
Schedule it
Run once, or on a cron. Every run refreshes PostgreSQL with the latest File URL / CSV data.
Do I need an add-on or extension for this?
No. The job runs server-side and writes into PostgreSQL through its API, so there is nothing installed in the destination itself. It keeps running when nobody has the file open, and a copy of the file does not need anything installed to work.
How often does the data refresh?
On whatever schedule you set with a cron expression — hourly, daily, or a specific time on specific days. Each run pulls the latest from File URL / CSV.
Do I need to write any code?
No. You connect both sides, map the fields in a wizard and set a schedule. Computed fields accept small expressions — abs, round, min, max, len — but there is nothing to host or maintain.
Does the file have to be public?
No. A public HTTPS link works with no credential, but you can also upload the file to your account storage and read it from there when it is not publicly reachable.
What happens if the URL redirects?
Up to five hops are followed, and the destination is re-validated at every one. A link that later starts redirecting somewhere internal does not quietly become a request to somewhere it should not go.
Can I run my own SQL rather than reading a table?
Yes. Set a query instead of a table and aggregate in the database, which is usually far cheaper than moving raw rows and aggregating downstream. One statement only - multi-statement queries are rejected.
Does it need a superuser?
No. A user with SELECT on what you are reading is enough for a source; a destination needs INSERT and, for upsert, UPDATE on the target table.
Is the File URL / CSV connector free to use?
You can connect File URL / CSV and start syncing on the free plan.
Other sources into PostgreSQL
Related reading
Airtable upsert: stop creating duplicate records
Appending on every run turns a tidy base into six copies of every row. Here is how upsert works in Airtable, and how to choose a merge key that holds up.
Import a CSV from a URL into Airtable
Point a job at a CSV link and it re-reads it on your schedule — no download, no manual import. Here is the setup, and the field-type trap to avoid first.
Get data into Google Sheets without an add-on
Every major Sheets data tool is an add-on you install, authorise and keep updated inside the spreadsheet. Here is how to schedule the same imports without one.