How it works

PromptShrink uses LLMLingua-2, a small model from Microsoft. It reads your text, gives every word a score for how much it matters, and removes the lowest-scoring words. It never rewrites or paraphrases: what's left is your own text, word for word, minus the filler.

Press Play to walk through the seven steps.

The model was trained by asking GPT-4 to compress thousands of meeting transcripts, then learning to copy its choices. The result often reads like a telegram, but large language models understand it almost as well as the original.

Edit the text or drag the slider. Struck-through words are the ones the model removes; highlighted parts are kept exactly.

When to use it

Settings explained

Strength (keep %)

The share of tokens to keep, from 10% to 90%. 50% makes the text about 2× smaller. Levels:

Token budget

Give a maximum token count instead, e.g. to fit a context window. PromptShrink works out the rate for you. If the text already fits, nothing is removed.

Model

Small (BERT-base, ~710 MB) is fast and good for most text. Large (XLM-RoBERTa-large, ~2.2 GB) makes slightly better choices and is slower. Both handle many languages.

Keep exactly

Finds numbers, links and code automatically and keeps them exactly as written.

Live update

After the first compression, changing a setting re-runs it straight away, so you can drag the strength slider and watch the result change.

Tidy spacing

LLMLingua rebuilds text with a space between every piece (“( Head of Product )”). Tidy spacing fixes that, which also saves a few tokens. On Settings.

Protected tokens

Single characters or words that are never removed. The defaults keep line breaks and sentence punctuation so the structure survives. On Settings.

Settings file

Port, Ollama model, defaults and limits live in .env in the app folder. Edit it, then run ./promptshrink.sh restart. The values in effect are listed on Settings.

Protecting text

Select text in the Original box and click Protect. It's wrapped in LLMLingua's tag and kept exactly:

<llmlingua, compress=False>Answer in JSON with keys id and total.</llmlingua>

You can also give one section its own rate, for example compress background more than the main text:

<llmlingua, rate=0.3>Long background section…</llmlingua>

Tags are removed from the output and from token counts. Sections can't be nested.

Reading the numbers

Token

The unit LLMs read and bill by, about ¾ of an English word. Counts here use OpenAI's cl100k_base tokenizer so runs are comparable; your provider may count slightly differently.

Smaller by

How much smaller the text got: 50% means half the tokens are gone.

Ratio

Original ÷ compressed. 2.0x means half the size.

Tokens kept

The share actually kept. It can differ from your setting: protected parts are never compressed, and the model rounds to whole words.

Cost

Input tokens × price per million tokens from pricing.json. Output tokens aren't affected by prompt compression.

Keyboard shortcuts

Use it from code

The app has a local HTTP API. This request uses your current settings:

Full interactive API docs: /api/docs

Privacy

• Compression runs entirely on this machine. Your text is never sent to an external service.
• The quality check talks only to Ollama on your own machine. Ollama cloud models are hidden and refused.
• Internet is used only to download the models once (from Hugging Face) and to load the chart library for Compare rates. Neither request contains your text.
• History and settings are stored in this browser's local storage. Clear them any time on Settings.