Documentation
One call, a sealed engine, a verdict. Everything a training team or its coding agent needs to take a forecast, act on it, and report what happened.
What the service does
Before you spend GPU time curating data toward a target domain, send a sample of the target and a sample of the candidate pool. The engine answers in seconds, on CPU, without touching your model. The corpus never leaves your machines; only samples travel, over HTTPS, processed in memory and never written to disk.
SELECTDO-NOT-SELECTNO-FORECASTEvery verdict carries a confidence tier (high, medium, low), a scope card, notes, a fingerprint of the exact text it was made on, and a forecast_id you use later to report the outcome. Every forecast is timestamped before your first training step. The receipt that ties forecast to outcome is the product.
Quick start
You need Python 3 and your API key from the dashboard. The reference client has no dependencies.
curl -O https://echo-program.com/client/echospace_pilot.py
export ECHOSPACE_URL=https://api.echo-program.com
export ECHOSPACE_API_KEY=es_pilot_...
python3 echospace_pilot.py selftest
python3 echospace_pilot.py forecast --target ./target_docs/ --pool ./pool_docs/ --label first-try \
--model-params 124000000 --keep 0.25 --language en --regime pretraining --table --out fc_first.json
python3 echospace_pilot.py select-local --forecast fc_first.json --pool ./pool_docs/ --keep 0.25 --out keep.json
python3 echospace_pilot.py outcome --forecast fc_xxxxxxxxxxxx --selected 1.91 --baseline 2.05selftestchecks the client's hashing against the service's test vectors. Run it once per machine.forecastdraws seeded random samples of whole documents from both folders, sends them, and prints the verdict. With--tableand aSELECTverdict, the saved file also carries the scoring table.select-localranks your whole pool on your own machine from that table and writeskeep.json. The pool is never uploaded.outcomereports the held-out result of the run under the forecast id. That is the receipt.
keep.json holds the indices of the pool documents to keep, in the order the client read them: files in sorted name order, walking folders recursively, then line order inside .jsonl files.
Documents and sizes
A document is one string. The client reads a folder recursively: every .jsonl line with a text field is one document, and any other file is one document. Over raw HTTP you send lists of strings.
More target data sharpens the reading. A target that arrives as a single document is accepted but noted; send one entry per document. English, whitespace-delimited text is the validated range; text without word boundaries is flagged in script_check.
Your model and your data
The forecast reads your data, not your model. It runs on CPU, in seconds, whatever you are about to train. What changes with scale is the evidence behind the reading: the service has been checked against real training runs up to the sizes on the scope card. Beyond them the verdict is still issued, the confidence drops one tier, and the response says why.
Larger models
- State the size you will train with
--model-params. Above 300 million parameters the response is marked as outside the validated range. - Most teams train a small proxy before a large run. Take the forecast on the pair you would feed the proxy, and report the proxy's result. A run of that size sits in or near the validated range, and its receipt is evidence on your own data.
- Then take the forecast for the large run and report that outcome too. Every receipt above the validated range extends it. That is what the pilot is for.
Mixture-of-experts models
Report the total parameter count in --model-params and the parameters active per token in --active-params. The scope card is judged on the total.
Image and text models
The service reads text. For a vision-language job, send the text side of each example as one document: the caption, the question with its answer, or the transcript of the page. Images are not sent. The validated case is one 256-million-parameter model in continued training.
Languages and code
- English is the validated language. State yours with
--language. - Any language written with spaces between words is read the same way and marked as not validated: the verdict is issued, one tier lower.
- Text without word boundaries, such as Chinese, Japanese or Thai, cannot be read reliably today, and the response reports it in
script_check. If your pipeline already segments the text into words, send the segmented form and use the same form for selection. - Source code is read as whitespace-separated tokens and has not been validated. Pass
--content codeand the response is marked accordingly.
Very large pools
- A forecast never needs the whole pool. The client samples about 5,000,000 words from it, whatever its size.
select-localreads the whole pool once, on your hardware, at several million words per second per core. It runs as one process and holds two numbers per document.- For pools too large for one process, use the sharded mode. A document's score depends only on the document and the table, so shards are scored in parallel and the cut is made once, afterwards. The result is the same whatever the number of workers and whatever order the shards finish in.
# one machine: every entry directly inside ./pool/ is a shard python3 echospace_pilot.py select-sharded --forecast fc_first.json --pool ./pool/ --keep 0.25 --out ./keep/ --workers 16 # many machines: score each shard where it lives (needs the forecast file; no key, no network) python3 echospace_pilot.py score-shard --forecast fc_first.json --shard ./pool/part-0007 --out scores/part-0007.npz # then bring the score files together (12 bytes per document) and make the cut python3 echospace_pilot.py select-merge --forecast fc_first.json --scores ./scores/ --keep 0.25 --out ./keep/
- A shard is any file or folder of documents. Its name is its id, so names must be unique.
./keep/gets oneNAME.keep.jsonper shard and akeep_manifest.json.keep_indicesin a shard's file count documents inside that shard, in the client's read order.- The cut holds one shard's scores in memory at a time. On a laptop it takes about ten seconds per hundred million documents, after scoring.
- The selection is recorded once on your ledger, under the forecast, with the number of shards.
- The sharded keep list is not document-for-document the one
select-localwould write for the same pool: both apply the same rule with the same seed, but draw their tie-breaking randomness differently. Use one mode per run.
Authentication
Every call carries the key as a bearer token. Keys are stored hashed on the service and revoked on request; one key per organisation. Send a User-Agent naming your tool; a bare default Python agent is rejected at the edge before it reaches the service.
curl -s https://api.echo-program.com/v1/me -H "Authorization: Bearer $ECHOSPACE_API_KEY" -H "User-Agent: my-pipeline/1.0"
{"key_id": "key_a6326f70", "org": "Acme Research", "daily_quota": 50, "used_last_24h": 2, "forecasts_total": 7}Forecast
/v1/forecastSamples in, verdict out. Counts as one engine call.
targetpoollabeltableSELECT, also return the scoring tablecontextmodel_params (int), keep_fraction (0 to 1), language (ISO code), regime (pretraining, continued, finetuning), content (text, code), active_params (int, for mixture-of-experts). Outside the validated range the confidence drops one tier and the response says why; the verdict does not change.provenanceclient, pool_total_documents, target_total_documents, sample_seed. Recorded on the receipt, not verified.curl -s https://api.echo-program.com/v1/forecast \
-H "Authorization: Bearer $ECHOSPACE_API_KEY" -H "Content-Type: application/json" -H "User-Agent: my-pipeline/1.0" \
-d '{"target": ["doc one ...", "doc two ..."], "pool": ["...", "..."], "label": "first-try",
"context": {"model_params": 124000000, "keep_fraction": 0.25, "language": "en", "regime": "pretraining"}, "table": true}'{
"forecast_id": "fc_7e2cc32e34e0",
"verdict": "SELECT",
"confidence": "medium",
"reason": "...",
"unstable": false,
"target": {"documents": 6, "chunks": 42, "words": 5376},
"pool": {"documents": 60, "chunks": 281, "words": 35968, "subsampled": false},
"notes": [],
"script_check": {"unspaced_fraction": 0.0},
"scope": {
"validated": {"model_size": "...", "keep_fraction": "...", "text": "...", "regime": "...", "selector": "..."},
"outside_validated_range": [],
"confidence_before_scope": "medium"
},
"fingerprint": {"target_sha256": "...", "pool_sha256": "...", "hasher": "es-h1", "engine": "pilot-0.2"},
"provenance": {"client": "my-pipeline/1.0", "pool_total_documents": 812000, "target_total_documents": 3100, "sample_seed": 0},
"engine": "pilot-0.2",
"elapsed_s": 0.31,
"scoring_table": {"format": "float32-le-base64", "buckets": 16384, "hasher": "es-h1", "sha256": "...", "values": "..."}
}unstabletrue means the two halves of the target disagree on the call; add target data before acting.scoring_tableis present only when asked for and the verdict isSELECT; otherwise it isnullwith a note.- Keep the response file.
select-localneeds it, and the fingerprint proves later which text the forecast was made on.
Selection
Two ways to turn a SELECT into a keep list. The local way is the one to use for any pool you would not want to upload.
/v1/selectionsRecord a keep list you computed locally from the scoring table. The pool never left your machine. Not an engine call.
forecast_idkeep_fractionpool_total_documentskepttable_sha256, keep_sha256client{"selection_id": "sel_1db5b923d18a", "forecast_id": "fc_7e2cc32e34e0", "forecast_verdict": "SELECT", "mode": "local", "recorded": {"keep_fraction": 0.25, "pool_total_documents": 812000, "kept": 203000, "table_sha256": "...", "keep_sha256": "..."}}/v1/selectServer-side keep list for a small pool. Takes the same body as a forecast plus keep_fraction (default 0.25) and an optional forecast_id to reuse a verdict; without one it runs the forecast first. Counts as one engine call.
SELECTreturns a targeted keep list (mode: "targeted").DO-NOT-SELECTreturns a uniform random keep list of the same size (mode: "random"). That is the recommended action, not a fallback.NO-FORECASTis refused with 409: the engine will not choose a keep list it cannot stand behind.
{"selection_id": "sel_...", "forecast_id": "fc_...", "verdict": "SELECT", "mode": "targeted", "keep_fraction": 0.25, "pool_documents": 4000, "kept": 1000, "keep_indices": [3, 7, 12, ...], "hasher": "es-h1", "engine": "pilot-0.2", "note": "..."}The scoring table and the hasher
The table is a plain published importance-weighting selector fitted on your own samples: one float per bigram bucket, sent as float32-le-base64 with its sha256. It says nothing about how the verdict was reached. To score a document, hash its word bigrams into buckets, average the table's values over them, and keep the top fraction of the pool by that score. The reference client does exactly this; the service's own selection agrees with it bit for bit.
/v1/hasherThe hashing a local scorer must reproduce, with test vectors. Not an engine call.
{"hasher": "es-h1", "buckets": 16384,
"rule": "tokens = re.findall(r'\\S+', text.lower()); h = zlib.crc32(token.encode('utf-8')) & 0xFFFFFFFF; bucket = (h[i] * 1000003 + h[i+1]) % buckets",
"test_vectors": [{"text": "The quick brown fox jumps over the lazy dog.", "word_hashes": [...], "buckets": [...]}, ...]}If you port the scorer into your own pipeline, run the test vectors before every selection. A hasher that drifts produces a keep list that is silently wrong. The version (es-h1) is stamped on every forecast, table and selection.
Outcome
/v1/outcomeReport what the training run did. This is the receipt; a forecast without an outcome teaches neither of us anything. Not an engine call.
forecast_idactedselected, random, both, nonemetric_nameheld-out lossselected_value, baseline_valuelower_is_betternotesmodel_params, active_params, train_tokens, keep_fraction, architecture{"outcome_id": "oc_a1f8bc1852a8", "forecast_id": "fc_...", "forecast_verdict": "SELECT", "forecast_agreed_with_outcome": true, "thanks": "recorded; this is the receipt"}Where you can, train both arms: the curated keep list and a random keep list of the same size, same model, same budget, and report the held-out metric of each. One arm is still worth reporting.
python3 echospace_pilot.py outcome --forecast fc_xxxxxxxxxxxx --selected 1.91 --baseline 2.05 \
--model-params 2600000000 --train-tokens 13000000000000 --keep 0.25 --architecture "decoder, 24 layers"Other calls
/v1/meYour key: organisation, quota, calls used in the last 24 hours, forecasts on record.
/v1/forecasts?limit=50Your forecasts, newest first, up to 500, each with its label, the number of outcomes reported, and the latest outcome. The same rows appear in the dashboard ledger.
/v1/healthService and engine version. No key needed.
The interactive OpenAPI reference is at https://api.echo-program.com/v1/docs and the schema at https://api.echo-program.com/v1/openapi.json.
Errors
forecast_id not found for this key/v1/select on a NO-FORECAST verdictRetry-AfterEngine calls run one at a time, and large uploads are admitted one at a time. A call that cannot start within about twenty seconds is answered with 503 and a Retry-After header instead of being left waiting; the reference client retries by itself. Retry on network failure is safe: a forecast that did not return was not recorded.
Data handling
- Sent: samples only, over HTTPS, held in memory for the call, never written to disk, never reused.
- Kept: document and word counts, the fingerprint of the sample, the verdict, timestamps, the sampling provenance, and any outcome you report. Never your text. Never the scoring table.
- Aggregated, anonymised forecast and outcome statistics may improve calibration and appear in our publications. Never partner-identifiable, never text.
Give your agent one paragraph. It does the rest.
Paste this into Claude Code, Cursor, Codex or any agent with a shell. It reads the plain-text reference, fetches the client, installs a skill that knows the workflow and the rules, and runs the self-test with your key.
Set up the Echo Program pilot in this repository. 1. Read https://echo-program.com/llms-full.txt. 2. Download https://echo-program.com/client/echospace_pilot.py into tools/ (or the scripts folder this repo uses). 3. Save https://echo-program.com/agents/SKILL.md as .claude/skills/echospace/SKILL.md (or append https://echo-program.com/agents/AGENTS.md to AGENTS.md if this repo uses that). 4. Put ECHOSPACE_URL=https://api.echo-program.com in the environment and ask me for ECHOSPACE_API_KEY; never commit the key. 5. Run "python3 tools/echospace_pilot.py selftest" and show me the result.
The skill, the AGENTS.md section, llms.txt and this page are generated from one source, so they say the same thing. Below is what the skill teaches.
Setup
curl -O https://echo-program.com/client/echospace_pilot.py # Python 3, no dependencies export ECHOSPACE_URL=https://api.echo-program.com export ECHOSPACE_API_KEY=es_pilot_... # from the member's dashboard; never commit it python3 echospace_pilot.py selftest # once per machine
Workflow
- Put target documents in one folder and candidate-pool documents in another. One file is one document; a
.jsonlfile is one document per line undertext. - Forecast:
python3 echospace_pilot.py forecast --target T/ --pool P/ --label NAME --model-params N --keep 0.25 --language en --regime pretraining --table --out fc_NAME.json. Always pass--model-params,--keep,--language,--regime: the scope card depends on them. - Read
verdictandconfidencefrom the saved JSON. Act by the verdict table below. Never override aDO-NOT-SELECTwith a targeted keep list. - On
SELECT:python3 echospace_pilot.py select-local --forecast fc_NAME.json --pool P/ --keep 0.25 --out keep_NAME.json.keep_indicesare pool document indices in the client's read order (sorted file names, recursive, then.jsonlline order). Build the training set from them. - If the pool is too large for one process, replace
select-localwithselect-sharded --forecast fc_NAME.json --pool P/ --keep 0.25 --out keep_NAME/ --workers N: every entry directly insideP/is a shard, and each gets its ownSHARD.keep.jsonwith indices local to that shard. - On
DO-NOT-SELECT: train on a uniform random sample of the pool of the same size. That is the recommendation, not a fallback. - On
NO-FORECAST: do not curate on this evidence. Add target data and forecast again. - After training, report:
python3 echospace_pilot.py outcome --forecast fc_... --selected <curated metric> --baseline <random metric> --metric "held-out loss" --acted both. Train both arms when the budget allows.
SELECTselect-local; keep the top fractionDO-NOT-SELECTNO-FORECASTRules
- Never upload a large pool:
forecastsends samples andselect-localscores the pool locally. Useselect(server-side keep list) only for small pools. - Never port the hasher without running the
/v1/hashertest vectors first; a drifted hasher yields a silently wrong keep list. - Minimums: target about 2,600 words in two or more documents; pool at least four times the target. A 422 tells you which is short.
- Quota: 50 engine calls per key per 24 hours (
forecastandselect).selections,outcome,me,hasherare free. - Keep every
fc_*.jsonandkeep_*.jsonwith the training run; the fingerprint ties the forecast to the exact text. - The service reads text only. For a vision-language job send the text side of each example as one document. For a mixture-of-experts model report the total parameter count in
--model-params. - Confidence
lowor a note about the validated range means the job is outside what the service has been checked on; say so in your report to the human. - Outputs are forecasts, not guarantees. The training decision belongs to the human; present the verdict, the confidence, and the notes.
Reference
- Full API reference: https://echo-program.com/llms-full.txt (also https://echo-program.com/docs)
- OpenAPI: https://api.echo-program.com/v1/openapi.json
- Reference client: https://echo-program.com/client/echospace_pilot.py
Questions: [email protected]