Text processing 4,995 crates

tokencountParallel CLI that counts GPT-style tokens across your codebase759MIT OR Apache-2.0Command line utilitiesDevelopment tools oxibonsai-tokenizerPure Rust BPE tokenizer for OxiBonsai (MeCrab-compatible)754Apache-2.0EncodingText processing asimov-repository-cliASIMOV Repository Command-Line Interface (CLI)751UnlicenseText processing spdfdiff_typesShared data model, diagnostics, provenance, and limits for semantic PDF diff tools.750MITText processing e9571_str_libA Rust crate for string processing, sorting, counting, and hashing749MITAlgorithmsCryptography loctokCount LOC (lines of code) & TOK (LLM tokens), fast.749MIT OR Apache-2.0Command line utilitiesDevelopment tools piper-plus-g2pMultilingual G2P (Grapheme-to-Phoneme) for TTS — eSpeak-ng free746MITMultimedia::AudioText processing torsh-textNatural language processing utilities for ToRSh deep learning framework744Apache-2.0ScienceText processing typoglycemiaA function to convert text to typoglycemic format with a Leet-speak variant. The function takes a string as input and returns a…743MITText processing rust-functionsA collection of Rust utility functions (starting with format_number).742MITText processingValue formatting mq-crawlerDirectory crawler for batch Markdown file processing739MITCommand line utilitiesText processing skanSkan is a Rust-native, Java Scanner-inspired library that provides type-safe, convenient methods for parsing various data types…738MITCommand line utilitiesDevelopment tools text-document-test-harnessShared test setup utilities for text-document crate tests736MPL-2.0Data structuresParsing tools asimov-construct-cliASIMOV Construct Command-Line Interface (CLI)736UnlicenseText processing neotomaA flexible, cached parser combinator framework for Rust.735MIT OR Apache-2.0Parsing toolsText processing decodeA High-Performance Query Tool for JSON, YAML and TOML731MITCommand line utilitiesDevelopment tools wesichain-promptRust-native LLM agents & chains with resumable ReAct workflows731Apache-2.0 OR MITTemplate engineText processing sqz-cliUniversal LLM context compressor — squeeze tokens from prompts, code, JSON, logs, and conversations731non-standardCommand line utilitiesText processing parser-cliCommand-line interface for extracting text from various file formats730MITCommand line utilitiesText processing asimov-config-cliASIMOV Configuration Command-Line Interface (CLI)729UnlicenseText processing pdmtHigh-performance, deterministic templating library for Model Context Protocol (MCP) applications with comprehensive todo…727MIT OR Apache-2.0Development toolsTemplate engine x12-stream-parserLow-level, iterator-based X12 EDI stream parser.726MIT OR Apache-2.0Parsing toolsText processing naming_utilsA utility library for generating naming conventions, pluralizing words, and rest api paths in Rust.726MITText processing supply-chain-trust-crate-000023An implementation of regular expressions for Rust. This implementation uses finite automata and guarantees linear time matching…725MIT OR Apache-2.0Text processing tortillaSomewhat syntax-aware text wrapping for source code and plain text documents724MIT OR Apache-2.0Command line utilitiesText processing tripletsA composable, deterministic text data pipeline for ML. Ingest, denoise, chunk, split, and sample multi-source corpora into…721MIT OR Apache-2.0AlgorithmsArtificial intelligence mark-htmlA simple and efficient Markdown to HTML parser written in Rust.719MITText processingWeb programming workspacer-documentA Rust library for parsing, organizing, and managing documents within multi-workspace environments. Optimized for speed and…714MITData structuresDevelopment tools gaze-assemblyPolicy-to-pipeline assembly for Gaze714Apache-2.0 OR MITText processing lethe-core-rustHigh-performance hybrid retrieval engine combining BM25 lexical search with vector similarity using z-score fusion. Features hero…710MITAlgorithmsData structures thread-languageLanguage definitions and parsers for Thread709AGPL-3.0-or-later AND MITArtificial intelligenceCommand line utilities rml-coreA simple N-gram language model implementation in Rust705MITAlgorithmsScience parser-webWeb API for extracting text from various file formats704MITText processingWeb programming::HTTP server llm-transpileHigh-performance LLM context bridge — token-optimized document transpiler704Apache-2.0Command line utilitiesEncoding distill-strip-ansiStrip, transform, and normalize ANSI escape sequences702MIT OR Apache-2.0Command line utilitiesText processing text-typesetTurns rich text documents into GPU-ready glyph quads698MPL-2.0Text editorsText processing texmanA Rust-based package manager for LaTeX696CC-BY-NC-4.0Text editorsText processing fractoryTrack intent and flow, preserve context, and reconstruct sessions — for mosaic thinkers.695MIT OR Apache-2.0Command line utilitiesDevelopment tools supply-chain-trust-example-crate-000024Fast multiple substring searching.695Unlicense OR MITText processing bible-ioA Rust library for working with Bible text data structures692MITParsing toolsText processing shabdakoshshabdakosh — Pronunciation dictionary with ARPABET/CMUdict support for svara phonemes687GPL-3.0-onlyText processing myanmar_utilA collection of tools for processing Myanmar text including syllable breaking and other utilities686MITText processing memory-mcpMCP server for semantic memory — pure-Rust embeddings, vector search, git-backed storage685MIT OR Apache-2.0Artificial intelligenceCommand line utilities infiniloom-engineHigh-performance repository context engine for LLMs - AST parsing, token counting, and secret detection684MITDevelopment toolsParsing tools toolchestEssential utility collection for Rust - the missing complement to itertools683MIT OR Apache-2.0AlgorithmsData structures adaptogenA library for normalizing outputs from different LLM providers683MITParsing toolsText processing docspec-markdown-writerStreaming Markdown (CommonMark) writer for DocSpec events — paragraphs and headings only682MITText processing satteri-mdxjsCompile MDX to JavaScript in Rust.681MITCompilersText processing asimov-ollama-moduleASIMOV Ollama module.678UnlicenseCommand line utilitiesText processing ja-furiganaJapanese furigana (ruby) lookup library — Lindera-based analyzer + data-driven dictionary/rules from furigana-dict677MITInternationalization (i18n)Text processing pdfrsPure Rust PDF toolkit: Markdown to PDF, merge/split/validate, Unicode/CJK, charts, thesis layout, CLI (pdfcli) and WASM — no…677Apache-2.0Command line utilitiesEncoding crlfixA cross-platform port of dos2unix written in Rust, converts text file line endings from CRLF to LF676MITCommand line utilitiesText processing fop-typesCore types for Apache FOP Rust implementation674Apache-2.0Data structuresText processing rig-memvidMemvid-backed persistent memory and lexical store for Rig agents.674MITDatabase interfacesText processing tokenmonsterGreedy tiktoken-like tokenizer with embedded vocabulary (cl100k-base approximator)673MITAlgorithmsText processing awful_dataset_builderBuild LLM-ready Q/A datasets from reference text-to-question mappings produced by Awful Knowledge Synthesizer.673CC-BY-SA-4.0Command line utilitiesDevelopment tools mecab-ko-dict한국어 형태소 사전 관리 - 바이너리 포맷, FST 검색, 연접 비용673MIT OR Apache-2.0Data structuresText processing jsonguardInput inspection and output sanitization for JSON/JSONL, CSV, and TSV — guards against formula injection, bidi-override,…672MITEncodingParser implementations awful_book_sanitizerCLI to clean up OCR-mangled book excerpts into readable text using OpenAI-compatible APIs.671CC-BY-SA-4.0Command line utilitiesText processing grapheme-utilsHandy utils for working with utf-8 [unicode] Extended Grapheme Clusters670MITText processing tdk_sozlukTDK Sözlük API verilerini çeken bir Rust kütüphanesi670Apache-2.0API bindingsText processing mailrs-cleanEmail content cleanup primitives: HTML sanitization, tracking-pixel detection, bulk/automated sender heuristics, quoted-reply…670Apache-2.0 OR MITEmailText processing rexifytrait based matcher alternative to Regex669MITText processing editor-coreA headless editor engine focused on state management, Unicode-aware text measurement, and coordinate conversion.668MIT OR Apache-2.0Text editorsText processing finetype-modelCandle-based transformer model for FineType668MITScienceText processing qmd-cliCLI for qmd - lightweight SOTA local search engine for AI agents667MIT OR Apache-2.0Command line utilitiesText processing catatauA TUI EPUB reader with fuzzy search and navigation666MIT OR Apache-2.0Command line utilitiesText processing mdlessA terminal-based markdown file viewer665Apache-2.0Command line utilitiesText processing redactorSecure PDF redaction library with Type3 font support using MuPDF662MITCommand line utilitiesText processing asimov-openai-moduleASIMOV OpenAI module.662UnlicenseCommand line utilitiesText processing pinyin2chA Rust library for converting Chinese Pinyin to Chinese characters with various levels of detail.661MITText processing fontsHigh-performance font parsing and analysis library for Grida Canvas661MITGraphicsText processing deep-diffA small crate to deeply diff serde_json::Value trees.660MIT OR Apache-2.0Data structuresText processing piiPII detection and anonymization with deterministic, capability-aware NLP pipelines.660MIT OR Apache-2.0Text processing dfmtA fully featured dynamic template drop in replacement for the format! macro.657MIT OR Apache-2.0No standard libraryTemplate engine inputx-pinyinSelf-developed Mandarin Pinyin input method engine — segmenter, fuzzy syllables, zero-dependency self-built finite-state index…657MIT OR Apache-2.0EncodingInternationalization (i18n) xberg-candle-ocrCandle-based VLM OCR engines for Xberg - pure-Rust transformer OCR (TrOCR, PaddleOCR-VL, GLM-OCR)656MITComputer visionText processing cli-image-to-asciiA library for rendering images as truecolor ANSI terminal art656MITCommand line utilitiesParser implementations character-traits-interpersonal-influenceA Rust crate providing fine-grained descriptors of interpersonal influence derived from historical and mythological archetypes.…654MITAlgorithmsScience mdxportMarkdown to PDF via Typst — comrak AST, in-process compilation, LaTeX math support652MITCommand line utilitiesText processing trustformers-tokenizersTokenizers for TrustformeRS651Apache-2.0ScienceText processing siftdb-cliCommand-line interface for SiftDB - the high-performance grep-native database650MITCommand line utilitiesDatabase interfaces super-swipe-typeA fast swipe typing prediction engine using ML649GPL-3.0-or-laterAlgorithmsText processing asimov-obsidian-moduleASIMOV module for integration with the Obsidian note-taking app.649UnlicenseCommand line utilitiesText processing speedgrepa simple grep tool648MPL-2.0AlgorithmsCommand line utilities notacySimple markdown note-taking tool645MITText processing anda_nexusAnda -- an AI Nexus powered by the Knowledge Interaction Protocol (KIP).645MIT OR Apache-2.0AccessibilityDevelopment tools docling-cliCommand-line interface for docling.rs (the `docling-rs` binary; a Rust port of docling).639MITCommand line utilitiesText processing litsea-cliLitsea is an extremely compact word segmentation and model training tool implemented in Rust.638MITText processing xberg-cliCommand-line interface for Xberg document intelligence637MITCommand line utilitiesText processing asimov-huggingface-moduleASIMOV module for model downloads from the Hugging Face Hub platform.635UnlicenseCommand line utilitiesText processing tsift-md-astFocused tree-sitter-md markdown AST + heading/section model — a leaf crate shared by tsift and other consumers (e.g. agent-doc)…635MITParsing toolsText processing serpscraperA CLI tool to fetch and convert search results into Markdown.634MITCommand line utilitiesText processing real-regexLinear-time, ReDoS-safe regular expressions with bounded lookarounds — Rust bindings to the REAL C++ engine.633MITText processing asimov-telegram-moduleASIMOV module for integration with the Telegram messaging service.630UnlicenseCommand line utilitiesText processing tmenuA simple TUI fuzzy finder630MITCommand line utilitiesText processing mailrs-attachment-extractExtract text from email attachments (PDF + image OCR). PDF text via `pdf-extract` (pure Rust); OCR via the `tesseract` CLI…629Apache-2.0 OR MITEmailFilesystem tool-attr-macroAttribute macro for defining tools in promptrs628LGPL-3.0Development toolsParsing tools ragzilla-parsingA Rust library for parsing PDFs using the Mistral AI OCR API627MITAPI bindingsText processing prettier-bytesA blazingly fast and safe, zero-allocation, `no_std`-compatible byte formatter.627MITEmbedded developmentNo standard library mailidatorA lightweight Rust library for checking email address misspellings626Apache-2.0Text processingWeb programming llm-repairJSON-repair, markdown-extraction, and tool-call recovery helpers for malformed LLM output625Apache-2.0 OR MITParser implementationsText processing xlrdA historical Excel BIFF8 xls format file reader in pure Rust624MITEncodingParsing tools zpl_toolchain_coreCore parser, emitter, and validator for ZPL II label code (part of the zpl-toolchain project)623MIT OR Apache-2.0Parsing toolsText processing asimov-apple-moduleASIMOV module for Apple devices.622UnlicenseCommand line utilitiesText processing redact-nerNamed Entity Recognition for PII detection using ONNX Runtime622Apache-2.0Text processingWebAssembly micropdfA pure Rust PDF library - A pure Rust PDF library with fz_/pdf_ API compatibility619MIT OR Apache-2.0Multimedia::ImagesParsing tools zettel-coreCore library for Luhmann-style Zettelkasten management610MITCommand line utilitiesText processing zhhzSelf-contained, static-build Simplified/Traditional Chinese converter — a pure-Rust, data-embedded reimplementation of OpenCC609Apache-2.0Command line utilitiesInternationalization (i18n) fop-coreCore FO tree parsing and property system for Apache FOP609Apache-2.0Parser implementationsText processing rustc-edit-distanceAn edit distance implementation derived from rustc, packaged for reuse.606MIT OR Apache-2.0AlgorithmsText processing rjotA minimalist, command-line jotting utility that's fast, private, and git-friendly.606MITCommand line utilitiesText processing rsre-luaa regex lua module using the fancy-regex rust crate602MITText processing radix_treesVarious radix tree (trie) data structures.601Apache-2.0Data structuresNo standard library nerdicons_rsAccess all NerdFonts Icons and Use It601MITText processingValue formatting csvpeek-rsA CLI tool to quickly peek into, list, and filter CSV data.601GPL-3.0-onlyCommand line utilitiesText processing oxifont-corePure Rust font trait surface and types600Apache-2.0Text processing asimov-nearai-moduleASIMOV module for data import from the NEAR AI developer platform.599UnlicenseCommand line utilitiesText processing orbok-apporbok application binary: backend bootstrap and GUI launch599Apache-2.0Development toolsGUI lexsimDictionary-free, multilingual lexical similarity engine with hybrid word segmentation (tokenize + Jaccard + BM25 + TextRank +…598MIT OR Apache-2.0Text processing hypembedPure-Rust BERT-compatible text embedding inference for local-first applications.598MIT OR Apache-2.0ScienceText processing miaA model inference framework.596MIT OR Apache-2.0ScienceText processing just_fmtJust a crate for formatting some things596MIT OR Apache-2.0Text processing rig-memory-policyBackend-agnostic memory-policy primitives (frame metadata, content-hash dedup) shared by Rig memory-store adapters.596MITText processing char-positionsIterate chars and their positions, i.e. line, column, and byte ranges595MITAlgorithmsNo standard library search-semanticallyEmbeddable semantic code search with multi-signal POEM ranking595MITDevelopment toolsText processing task-shAI-powered shell command generator592MITCommand line utilitiesText processing anno-libNER, coreference resolution, relation extraction, PII detection, and zero-shot entity types592MIT OR Apache-2.0Artificial intelligenceText processing probe-codeAI-friendly, fully local, semantic code search tool for large codebases586MITCommand line utilitiesDevelopment tools llm-toolkit-expertise⚠️ DEPRECATED: Use llm-toolkit::agent::expertise instead. This crate is archived and will not receive updates.586MITText processing pdf2mdPDF → Markdown extractor with figure rasterization, table & banner detection. Built on pdfium-render.586MITCommand line utilitiesParsing tools rs-wikipages2structConverts Wikipedia pages into Rust structs.584Apache-2.0Command line utilitiesParsing tools looneygrepA powerful command-line search tool with web page support and replace functionality.584MIT OR Apache-2.0Command line utilitiesText processing TextToEmojiA library for converting words to emoji representations580MITText processing pdf_contentPDF content stream tokenizer and operator interpreter for semantic PDF diff tools.580MITParser implementationsText processing ripsed-coreCore edit engine for ripsed — pure logic, no I/O578MIT OR Apache-2.0Command line utilitiesText processing sparse-ngramsFast sparse n-gram extraction from byte slices.576MITAlgorithmsData structures llm-guardZero-copy guardrails for LLM input/output. Pure-Rust scanners (prompt-injection, role-override, secret leakage, PII, invisible…573MIT OR Apache-2.0Text processing docxide-pdfLibrary and CLI for converting DOCX files to PDF, matching Microsoft Word's output as closely as possible573Apache-2.0Command line utilitiesText editors fopFOP (Formatting Objects Processor) — Apache FOP-compatible XSL-FO processor in pure Rust572Apache-2.0Parser implementationsRendering ada-idnaFast IDNA (Internationalized Domain Names in Applications) implementation in Rust570MIT OR Apache-2.0Text processingWeb programming curlicueHelix keybinding utilities570MITText processing niblitsToken-aware, multi-format text chunking library with language-aware semantic splitting569MITAlgorithmsAsynchronous kowalski-cliKowalski CLI Interface: A Rust-based agent for interacting with Ollama models569MITNetwork programmingScience hydroperx-utf16Work with UTF-16 in Rust568Apache-2.0EncodingText processing thread-servicesService layer interfaces for Thread568AGPL-3.0-or-laterArtificial intelligenceCommand line utilities hjkl-fuzzySubsequence fuzzy scorer (score + match positions) for the hjkl editor stack.568MITText processing aicx-parserTranscript parser, chunker, and slicer for AI session transcripts (Claude, Codex, Gemini, Junie). Companion crate of aicx.567BUSL-1.1Parsing toolsText processing nodedb-queryShared query evaluation engine for NodeDB — expressions, filters, functions, aggregations, window functions566Apache-2.0AlgorithmsText processing decollateBlazing-fast, locale-aware, context-free string collation566MITInternationalization (i18n)Text processing ucp-translator-markdownMarkdown translator for UCP566MITText processing oxipdf-irIntermediate representation types for the oxipdf PDF engine565MIT OR Apache-2.0RenderingText processing mq-checkType checker for mq565MITCommand line utilitiesText processing arborium-cliCommand-line syntax highlighter powered by arborium564MIT OR Apache-2.0Command line utilitiesText processing rsword_chirhoCore SWORD module library in pure Rust563GPL-2.0-or-laterText processing jmdict-fastBlazing-fast Japanese dictionary engine with FST-based indexing557MITInternationalization (i18n)Text processing aimem-coreCore library for AiMem — give your AI a memory. Hybrid search + knowledge graph over Turso.555MITDatabase interfacesText processing oxifont-parserPure Rust TTF/OTF/TTC font parser for OxiFont (ttf-parser wrapper)554Apache-2.0Parser implementationsText processing wg-ragsmithSemantic chunking and RAG utilities for document processing and retrieval-augmented generation.553MITAlgorithmsAPI bindings redstrRed team string obfuscation and transformation for offensive security, WAF bypass, XSS, SQL injection, phishing, and evasion…549MITDevelopment toolsEncoding md-cli-testA helper library for integration testing of CLI apps using markdown files as a source of test cases548MITCommand-line interfaceDevelopment tools::Testing e2mA lightweight CLI tool that transforms English words into emojis using only Rust’s standard library.547MITCommand line utilitiesText processing slabsRetrieval spans and late pooling547MIT OR Apache-2.0Text processing runefix-cliUnified CLI frontend for runefix tools: width detection, markdown formatter, and terminal testing546MIT OR Apache-2.0Command line utilitiesText processing markdown-live-previewA live Markdown preview server with WebSocket and HTML rendering542MITText processingWeb programming fop-layoutLayout engine for Apache FOP Rust implementation541Apache-2.0RenderingText processing pdf_textPositioned PDF text and glyph extraction for semantic diff and comparison pipelines.541MITText processing citumCitum CLI: render, check, convert, and manage citation styles, references, and documents540MIT OR Apache-2.0Command line utilitiesText processing asimov-vcard-moduleASIMOV module for vCard (VCF) file format support.536UnlicenseCommand line utilitiesText processing shabdashabda — Grapheme-to-phoneme (G2P) conversion: text to phoneme sequences for vocal synthesis536GPL-3.0-onlyMultimedia::AudioText processing sa3p-parserStreaming XML-style parser for SA3P operator commands.536MITParser implementationsText processing pdf_coreLow-level PDF parser foundation for semantic PDF diff and comparison tools.535MITParser implementationsText processing rjxA fast and lightweight JSON processor and query tool in Rust531MITCommand line utilitiesParsing tools hwpforge-bindings-mcpAnvil — HwpForge MCP server for AI-native HWPX document tools528MIT OR Apache-2.0EncodingInternationalization (i18n) hardly-trieA Rust library for efficient trie data structure implementation.523ZlibAlgorithmsData structures graphrag-coreCore portable library for GraphRAG - works on native and WASM523MITAlgorithmsScience shuck-formatFormatting primitives for shuck523MITDevelopment toolsText processing zaliznyakA Russian inflection library522MIT OR Apache-2.0Data structuresInternationalization (i18n) quotA fast and flexible command-line tool that converts text input into escaped string literals520Elastic-2.0Command line utilitiesText processing structured-email-addressRFC 5321/5322/6531 email address parser, validator, and normalizer. Subaddress extraction, provider-aware normalization, PSL…517Apache-2.0Parser implementationsText processing vectorlessReasoning-based Document Engine512Apache-2.0AlgorithmsData structures roberto-mcpRoberto MCP - A lightning-fast, language-agnostic code analysis MCP server built in Rust510Apache-2.0Development toolsParsing tools perl-line-indexByte-oriented line/column index for incremental parsing508MIT OR Apache-2.0Development toolsParsing tools mecab-ko-core한국어 형태소 분석 핵심 엔진 - Lattice, Viterbi, 토크나이저507MIT OR Apache-2.0ScienceText processing jjpwrgem-parseJSON parser and formatter with rich errors507MITParser implementationsText processing acdc-parser`AsciiDoc` parser using PEG grammars506MIT OR Apache-2.0Parsing toolsText processing shellveticaBring your shell output to the web506GPL-3.0-or-laterCommand line utilitiesText processing arabic_pdf_to_textA CLI tool to convert Arabic PDFs to text using Google's Gemini API501MITCommand line utilitiesText processing textconStreaming file bundling and one-pass template references for AI/LLM context500EUPL-1.2Command line utilitiesText processing asimov-outlook-moduleASIMOV module for Outlook email access.498UnlicenseCommand line utilitiesText processing nihoA command-line tool for converting romanized Japanese text to Japanese characters498MITCommand line utilitiesText processing numaelis-rckive-genpdfUser-friendly PDF generator written in pure Rust498EUPL-1.2Text processing gatekpr-patternsRegex pattern registry and pre-built pattern sets for Shopify validation497MITText processing figureheadA Rust library to convert Mermaid.js diagram markup into ASCII diagrams496MITGraphicsParsing tools ucp-llmLLM-focused utilities for the Unified Content Protocol496MITCommand line utilitiesText processing regextrieHigh-performance Rust library designed for matching a large number of regular expressions against a given input string495non-standardAlgorithmsParsing tools semantic-code-edit-mcpMCP server for semantic code editing with tree-sitter494Apache-2.0Development toolsText processing thread-rule-engineRule-based scanning and transformation engine for Thread494AGPL-3.0-or-later AND MITArtificial intelligenceCommand line utilities xxxxx_rust_stsA collection of useful string and file utilities for Rust494MITDevelopment toolsText processing xore-configXORE 配置管理模块 - 统一的配置加载和路径管理494GPL-3.0Command line utilitiesData structures