solrfal for TYPO3: File Indexing with Apache Solr

Why the TYPO3 default search fails at scale with document archives

As soon as a TYPO3 installation manages more than a few hundred PDF documents, the built-in search runs into two hard limits: it only finds content stored in database fields, not inside files, and its ranking logic ignores the relevance signals that editors actually need. solrfal closes exactly this gap by wiring the TYPO3 File Abstraction Layer (FAL) directly into Apache Solr and making every file searchable, including its metadata. The extension is aimed at organisations that treat documents as the core of their knowledge work: public authorities with forms, publishers with specialist literature, universities with teaching materials.

Typical use cases

At a technical university with around 40,000 students, the examination regulations live in 180 PDF files distributed across twelve faculties. Without solrfal, a student can only find the paragraph about registering for an exam if an editor has additionally copied the text into a meta description. With solrfal, Apache Tika indexes every page and the search query returns the right paragraph including the citation.

A second scenario is familiar to government agencies that publish laws, guidelines and forms through a TYPO3 CMS. The documents change frequently and access rights are tied to organisational units. solrfal automatically synchronises the fe_groups membership of each file into the Solr index, so that a staff member from department V only sees results they are authorised to read.

The third case appears at specialist publishers: a publisher with 12,000 journal articles as PDF wants to make its archive searchable via a faceted search by year, author and category. solrfal extracts the metadata, populates the Solr fields and provides the data basis for a facet that classic TYPO3 extensions such as ke_search can no longer handle at this scale.

Technical architecture on top of Apache Tika and EXT:solr

solrfal is an extension of the core EXT:solr extension from dkd Internet Service and requires a running Apache Solr server, typically version 8 or 9. The actual text extraction is handled by Apache Tika, either as an embedded service inside the Solr container or as a standalone Tika server. solrfal hooks into the FAL lifecycle via the TYPO3 event API: every uploaded, moved or deleted file triggers an indexing job that is processed asynchronously by the TYPO3 scheduler.

Configuration runs through TypoScript and the extension configuration. The Solr schema can be adjusted via the Managed Schema API, so additional fields such as document type, department or language can be added without a Solr restart. Relevance tuning happens via boosting queries and function queries defined in the EXT:solr query configuration. solrfal inherits all language features of EXT:solr, including multilingual analyzers for English, German, French and other standard languages.

Common problems and solutions

The first problem usually surfaces during the initial index run: Apache Tika crashes with an OutOfMemoryError on broken or encrypted PDFs and takes the entire indexer down with it. The extension then marks the file as faulty but does not automatically skip it on the next run. We analyse the Tika logs, separate the encrypted files from the genuinely damaged ones and set up a pre-check that filters problematic files out before they ever reach the indexer.

The second recurring topic is relevance. Teams report that the search does find all documents, but irrelevant hits appear at the top. The cause is almost always the default field weighting: solrfal indexes the entire file content into a single field without distinguishing between title, headings and body. A clean solution requires a tailored schema with separate fields for title, metadata and body, plus boosting rules that weight title matches higher.

The third problem concerns performance with large archives. From around 50,000 indexed files onwards, the scheduler becomes the bottleneck because solrfal processes every job individually. The solution is batch indexing combined with a dedicated worker process that runs in parallel to the normal scheduler tasks and processes solrfal jobs with priority. On top of that, differential indexing pays off: only newly added or changed files are reprocessed on each run, which significantly reduces maintenance overhead for stable document archives.

A fourth, rarer issue appears in multilingual installations: solrfal indexes files language-agnostic into a shared index, so a French study can show up in the German results list if the search term is internationally common. A clean language separation requires multiple Solr cores or an additional language facet, which can be configured per site root via the EXT:solr configuration.

Migration and version compatibility

solrfal follows the release cycle of EXT:solr, which currently supports TYPO3 v12 and v13. The jump from TYPO3 v9 to v12 is the most common migration path and almost always means a jump from Solr 6 to Solr 9 as well. The schema format, the Managed Schema API and several analyzer classes change along the way, which makes a full reindex mandatory. Existing boosting rules have to be validated against the new query parser behaviour, because Solr 9 evaluates certain default operators differently than Solr 6.

Anyone migrating from ke_search or a purely database-backed search has to factor in that solrfal requires a dedicated Solr server and therefore extends the hosting requirements. Gosign has been guiding these migrations for years and, where needed, also handles the switch to a containerised Solr setup that fits into existing deployment pipelines.

It is also worth noting that the maintenance burden of solrfal should not be underestimated: Solr itself receives regular security updates, and the schema has to be reviewed against new analyzer classes on every major upgrade. A project that commits to solrfal also commits to running its own search stack and should factor that into the initial sizing and into the planning of operational resources.