atproto2elasticsearch 1.0.0

Thought it’s time to declare atproto2elasticsearch stable. It’s a CLI that connects to the Bluesky (AT Protocol) firehose, converts posts into NDJSON, and indexes them into Elasticsearch using node-es-transformer. Real-time firehose data, searchable in ES.

The 1.0.0 release is mostly polish: better structure, documentation, and a small programmatic API for reuse.

What changed

CLI + programmatic API

The CLI is still the main entry point, but there’s now an exported API for programmatic use (startIngestion, loadConfig, defaultMappings). Example:

const { startIngestion, loadConfig, defaultMappings } = require("atproto2elasticsearch");

const config = loadConfig({
  ES_NODE: "https://user:pass@localhost:9200",
  ES_INDEX: "bluesky-firehose-0001",
});

startIngestion({ config, mappings: defaultMappings });

Config options

Added a few more env vars so you can control TLS verification, ingest pipelines, logging, and progress output:

• ES_TLS_REJECT_UNAUTHORIZED
• ES_PIPELINE
• LOG_LEVEL
• PROGRESS

Documentation + examples

There is now a full doc set (CONTRIBUTING, SECURITY, DEVELOPMENT, TESTING, MIGRATION, VERSIONING, etc.) plus runnable examples and a docker-compose file to spin up Elasticsearch locally.

Tests + CI

Added node:test coverage for config parsing and default mappings, plus type checking and CI verification.

Node.js 22 minimum

Still on Node 22+ to keep the dependency stack modern.

Upgrading from alpha

No breaking changes to the CLI. If you already run it, you can upgrade directly as long as you’re on Node 22+. See MIGRATION.md if you need the details.

That’s it. It’s a small tool I use for my own Bluesky search experiments, now cleaned up enough to call it stable. Install with npx atproto2elasticsearch or check out the GitHub repo for docs and examples.

Most of this work was done with pi-coding-agent, and yes, it saved me from writing a dozen Markdown files by hand.