Information extraction is often treated as a one-pass prediction task: give a model a document, ask it to extract entities or relations, and the resulting knowledge is not reused to improve future extraction.
That setup doesn’t work well when the domain keeps changing. In medical, legal, HR, scientific, and enterprise knowledge settings, terminology evolves, rare cases matter, and taxonomies evolve. A static extractor can miss emerging language, carry over previous errors, or require expensive adaptation cycles.
At ACL 2026 in San Diego, we introduce DySECT, the Dynamic Self-Evolving Extraction and Curation Toolkit. The central idea is straightforward: extraction systems should not only produce structured knowledge, but also should exploit the structured knowledge in turn for extraction.
Extraction as a Closed Loop
DySECT transforms information extraction into a feedback loop.
DySECT starts with an LLM extracting concept-level triples from raw text (step 1). Extracted triplets are inserted into a knowledge base (KB) (step 2). The KB then consolidates the extracted knowledge, retrieves extra relevant information from external sources (LLMs, Web, etc.), assigns confidence scores, organizes concepts into more meaningful hierarchies, and identifies conflicts such as mutually exclusive concepts or relations (step 3).
That structured knowledge is then fed back into the extractor (step 4), improving the quality of the next extraction. This is the self- evolving loop between extractor and KB.
This can happen through prompt augmentation, sampled examples, hierarchical concept cues, or optional synthetic data generated from high-confidence KB content. In other words, each extraction pass gives the system more explicit knowledge to guide the next one, evolving and fine-tuning the KB.
The result is not simply a larger prompt. It is an adaptive extraction architecture where accumulated knowledge becomes a reusable system component, allowing previously extracted concepts to be evaluated and evolve as the knowledge base grows.
Why Explicit Knowledge Matters
A core design choice in DySECT is that knowledge remains inspectable.
Many adaptation strategies rely on retraining, fine-tuning, or implicit parameter updates. Those approaches can be powerful, but they also make it difficult to see what the system has learned, correct specific errors, or enforce domain policies.
DySECT instead keeps accumulated knowledge in an explicit KB. Users can inspect triples, review confidence statistics, examine hierarchical structure, and manually validate, invalidate, or add knowledge when needed.
That matters for enterprise AI. In domains where extraction supports search, ranking, compliance, analytics, or decision support, teams need more than incremental accuracy gains. They need to understand what’s the confidence in the knowledge piece, where that knowledge came from, and how it can be corrected.
Megagon's Contributions
DySECT combines three mechanisms into a self-evolving extraction framework.
First, the extraction step uses an LLM to produce structured triples from raw text.
Second, the knowledge base grows over time. It integrates new evidence, estimates confidence, clusters semantically related concepts, creates intermediate abstractions, and applies lightweight constraints such as mutual exclusivity to discourage logically inconsistent extractions and reflecting it in their confidence score.
Third, the KB feeds knowledge back into the extractor. High-confidence examples, long-tail concepts, hierarchical categories, and negative cues can all shape later extraction iterations.
The system supports both positive and negative feedback modes. Positive feedback guides the extractor toward semantically consistent regions of the KB. Negative feedback discourages already-covered or saturated patterns, encouraging the system to explore underrepresented areas.
This creates a useful trade-off: positive feedback can support stability and schema alignment, while negative feedback can support discovery and coverage expansion.
Key Findings
The paper evaluates DySECT on DocRED, a document-level relation extraction benchmark built from Wikipedia and Wikidata.
Across GPT-4.1, GPT-4.1-mini, LLaMA-3.3 70B, and Kimi K2.5, KB-guided extraction consistently improves recall over baseline extraction without KB feedback.
The first KB-guided iteration improves recall by 5-8% without synthetic data generation or fine-tuning. By the second iteration, recall continues to increase across the evaluated models.
For example, DySECT with GPT-4.1 improves from %22.80 recall at baseline to 30.62 after one positive-feedback iteration and 37.03 after the second. Kimi K2.5 improves from 32.03 at baseline to 37.85 and then 44.41 under positive feedback.
This finding points to an important insight: it shows that structured knowledge reuse alone can improve extraction quality. The model does not need to be retrained for the system to become more domain-aware over time.
The Broader Lesson
LLM systems can improve through explicit, governed knowledge base.
For information extraction, this changes the operational model. Instead of rebuilding an extractor whenever terminology shifts, a system can accumulate domain knowledge during use, organize it into abstractions, and reuse it in later extraction.
This is especially relevant for enterprise AI systems that operate in changing environments. Product catalogs evolve. Job titles shift. Clinical and legal language changes. Internal taxonomies drift. New entities and relations appear before teams have time to create clean training data.
DySECT offers one path toward extraction systems that adapt while preserving visibility and control.
Practical Implication
Self-evolving extraction, as demonstrated in the DySECT version presented at ACL2026, does not yet solve every adaptation problem. That framework still depends on extractor quality and prompt design, KB reuse may vary by domain, noisy or biased triples may degrade quality, and iterative extraction adds computational overhead.
But the DySECT shows the feasibility of post-deployment self-improving systems.
Rather than relying solely on larger models or offline retraining cycles, DySECT shows how structured knowledge, confidence modeling, and human oversight can become part of the extraction loop itself.
For teams building LLM-based data systems, here’s your takeaway: extraction quality is not only a model problem, it is also a knowledge, governance, and feedback problem, and it can continue to improve even after the system is deployed in production.
Read the paper: A Dynamic Self-Evolving Extraction System
Written By: Moin Aminnaseri and Megagon Labs
