skip to content
walterra.dev
Table of Contents

Thought it’s time to give node-es-transformer the honour of hitting 1.0.0 after sitting in alpha/beta for years. It’s a Node.js library for ingesting largish CSV/JSON files into Elasticsearch using streams, which means you can process multi-gigabyte files without running out of memory.

I built it years ago, the 1.0.0 release is really more like a maintenance release with some housekeeping and improved Elasticsearch version support.

What changed

Elasticsearch 8.x and 9.x support

The main addition is dual-version support. The library now works with both Elasticsearch 8.x and 9.x, and can reindex between them:

await transformer({
  sourceClientConfig: { node: 'https://es8-cluster:9200' },
  targetClientConfig: { node: 'https://es9-cluster:9200' },
  sourceIndexName: 'my-source-index',
  targetIndexName: 'my-target-index',
});

It detects the version automatically by querying the cluster. The implementation uses npm package aliases to install both client versions and picks the right one at runtime.

TypeScript definitions

Added proper TypeScript definitions so you get autocomplete and type checking if you’re using it from TypeScript. Nothing fancy, just the usual .d.ts file.

Node.js 22 minimum

Bumped the minimum Node version to 22. If you’re still on 18 or 20, you’ll need to stick with the beta releases or upgrade.

Other changes

  • Switched to Changesets for version management (automated releases are nice)
  • Modernized the test setup to use testcontainers instead of requiring a local ES cluster
  • Added a bunch of documentation files (CONTRIBUTING.md, MIGRATION.md, etc.)
  • Updated dependencies (glob v7→v13, jest v29→v30)
  • Fixed some test warnings about open file handles

Upgrading from beta

If you’re coming from an earlier beta version, the main thing is you need Node.js 22+. The API is otherwise the same. See MIGRATION.md if you run into issues.

That’s about it. It’s a side project I maintain when I have time, but it’s been useful to me and hopefully to others. Install with npm install node-es-transformer or check out the GitHub repo for examples and documentation.

Most of this work was done with pi-coding-agent using Claude Opus 4.5. It went pretty smoothly, though there was some swearing involved (at the model, not the coding agent).