By Township Canada
Why Your 5,000-Record Batch Conversion Was Timing Out (And Why It Won't Anymore)
Large CSV uploads to Township Canada's batch converter were failing with 504 errors. A set-based rewrite of the database layer fixes it: 5,000 records now complete in seconds.
You upload a CSV with 2,000 LSD records for an AER well licence review. Three minutes later the batch converter returns a 504 gateway timeout. The job did not finish. You split the file into four chunks of 500 and run each one separately. Four uploads, four waits, four result sets to stitch back together. It works, but the tool was supposed to handle this in a single pass.
That workaround is no longer necessary. Township Canada's batch converter now processes up to 5,000 records in seconds, not minutes, and without the timeout failures that forced users to split large files.
What a 504 Actually Meant
A 504 gateway timeout means the server took longer than the platform allows and the request was killed before it could respond. For the batch converter, the cause was not query complexity. The queries themselves were fast: each legal land description lookup hit an indexed PostGIS table and returned in roughly 10 milliseconds. The problem was volume.
The original implementation looked up each record individually. For every row in the CSV, the server made a separate round trip to the database: send the query, wait for the response, move to the next row. At 10 ms per round trip, a 2,000-record batch accumulated about 20 seconds of pure database latency before any enrichment happened. A full 5,000-record job reached roughly 66 seconds of round-trip time, and the platform killed the function at 60 seconds.
The bottleneck was not the database. It was the number of times the server talked to it.
What Changed
The batch converter now uses set-based queries instead of per-record loops. Here is what that means in practice:
Deduplication. If a CSV contains the same legal land description on multiple rows (common in regulatory filings where one well site appears in several compliance records), the converter identifies the duplicates and queries each unique description once. The results are mapped back to every row that referenced it.
Chunked queries. The deduplicated list is split into chunks of 500 records. Each chunk is sent to the database as a single query that returns all 500 results at once. A full 5,000-record batch costs about 10 database round trips instead of 5,000.
Graceful degradation. If one chunk fails (a temporary database hiccup, a malformed record in that batch), only those records come back as misses. The rest of the job completes normally. Previously, a single error in the middle of the sequential loop could cascade and fail the entire batch.
Efficient enrichment. When you enable enrichment options (administrative boundaries, oil and gas fields, soil characteristics), the enrichment queries now run against each chunk as a batch rather than per record. The spatial lookups still use per-row index probes on the underlying PostGIS tables, which keeps them fast, but they run inside the chunked query instead of in a separate loop.
Parameterized queries. GPS coordinates are now passed as bound parameters in the SQL, not interpolated into the query string. This is a security improvement that also makes the query planner's job easier.
The net effect: a 5,000-record batch that used to time out now finishes well inside the server's timeout window. Most jobs complete in single-digit seconds.
What This Looks Like for a Lease Block Conversion
Consider a concrete scenario. You are a land technician preparing a well licence application for a Duvernay horizontal well in west-central Alberta. The AER filing requires GPS coordinates for every legal subdivision in the lease block and surrounding offset wells. Your spreadsheet has 1,800 rows of DLS locations in the format 14-27-048-05W5, one per row, with columns for well name, operator, and licence number.
Before this fix, uploading that CSV to Township Canada's batch converter was unreliable above roughly 1,000 rows. You might get a result. You might get a 504. The only safe approach was to split the file and run multiple uploads. After the conversion, you had to merge the result files back together before you could export to Shapefile or KML for the regulatory submission.
Now you upload the full 1,800-row CSV once. The converter detects the legal_land_description column, previews the first few rows, and runs the conversion. Every row comes back with GPS coordinates, province, and municipality. If you enabled the oil and gas fields enrichment option, each record also carries the field name where the well sits. You export the full result set to Enhanced CSV (which preserves your original columns alongside the converted output), or to Shapefile for direct import into ArcGIS.
The same improvement applies to the reverse direction. If you have a spreadsheet of GPS coordinates from field GPS units or drone surveys and need the legal land description for each point, the /api/batch/coordinates endpoint and the web UI both use the same set-based lookup.
Who Else Benefits
The oil and gas use case is the most common large-batch scenario, but it is not the only one.
Crop insurance adjusters at AFSC or SCIC process claim declarations after hail storms, drought events, or flooding. A single weather event can affect hundreds of quarter sections in a region. The adjuster exports the affected legal land descriptions from the claims system, needs coordinates for mapping, and needs the job done before the adjustment deadline. A 504 on a 1,200-record batch meant splitting the file and losing time that should have gone to fieldwork.
Land agents and brokers running bulk due diligence across multiple townships. A farmland acquisition pipeline review might involve 500 to 2,000 legal land descriptions across Alberta and Saskatchewan, all needing coordinates and enrichment data for soil type and administrative boundaries. The batch conversion guide covers how to set up the CSV and choose enrichment options for these scenarios.
Pipeline corridor planners converting linear reference lists. A proposed pipeline route generates a list of every legal subdivision it crosses, sometimes over 100 km and several hundred LSDs. Converting those to GPS coordinates and exporting to KML shows the corridor on a map for the environmental assessment.
Developers calling the batch API from internal tools. The /api/batch/legal-location and /api/batch/coordinates endpoints both use the set-based lookup. If you had implemented client-side chunking to work around the timeouts, that code is no longer needed (though it will not hurt anything if left in place).
How to Use It
Nothing has changed in how you submit a batch. Upload a CSV at townshipcanada.com/app/batch or send a POST request to the batch API endpoints. The converter still accepts up to 5,000 records per job, still auto-detects whether you are converting legal land descriptions to GPS or GPS to legal land descriptions based on your column headers, and still offers the same three enrichment options: administrative boundaries, oil and gas fields, and soil characteristics.
The difference is that the full 5,000-record limit is now reliable. You do not need to stay under an informal safe threshold or split files to avoid timeouts.
Batch conversion is available on the Business plan. If you are on a Pro or Starter plan and need to convert more than one record at a time, Business includes unlimited batch conversions, CSV upload, and all export formats (Enhanced CSV, KML, Shapefile, DXF, GeoJSON). For developers, the batch API is available as a separate API subscription.