• HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
Tuesday, November 17, 2026
BIOENGINEER.ORG
No Result
View All Result
  • Login
  • HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
        • Lecturer
        • PhD Studentship
        • Postdoc
        • Research Assistant
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
  • HOME
  • NEWS
  • EXPLORE
    • CAREER
      • Companies
      • Jobs
        • Lecturer
        • PhD Studentship
        • Postdoc
        • Research Assistant
    • EVENTS
    • iGEM
      • News
      • Team
    • PHOTOS
    • VIDEO
    • WIKI
  • BLOG
  • COMMUNITY
    • FACEBOOK
    • INSTAGRAM
    • TWITTER
No Result
View All Result
Bioengineer.org
No Result
View All Result
Home NEWS Science News Technology

Local Vision-Language System Retrieves Heritage Wine Labels Using Segmented Hybrid Search

by
November 12, 2026
in Technology
Reading Time: 6 mins read
0
Local Vision-Language System Retrieves Heritage Wine Labels Using Segmented Hybrid Search

Local Vision-Language System Retrieves Heritage Wine Labels Using Segmented Hybrid Search

Share on FacebookShare on TwitterShare on LinkedinShare on RedditShare on Telegram

Researchers have introduced WineEyes, an open-source software architecture designed to resolve the longstanding challenge of retrieving digitized historical ephemera using natural-language queries. Cultural archives frequently house massive collections of printed materials, exemplified by the Francisco Odero Álvarez collection of over 25,000 historical wine labels, which present significant cataloguing difficulties. While conventional relational databases effectively support lookups across predetermined metadata fields such as producer, region, or vintage, they cannot accommodate expressive iconographic requests describing complex compositions, such as searching for a woman with flowers in an art nouveau style. Historical wine-label research has largely focused on text recognition and instance matching, while digital humanities projects using vision-language spaces like CLIP often miss fine printed text and compositional nuances. To overcome these barriers without large domain-specific training sets or cloud services, WineEyes operationalises caption-mediated retrieval, converting visual compositions into auditable textual records searchable via local dense and sparse indexing.

The ingestion pipeline within WineEyes is structured across five sequential stages designed to transform raw photographic scans into structured, searchable records. In the initial transcription phase, the glm-ocr:latest vision model processes the uncropped photograph through a locally hosted Ollama instance to extract visible text before geometric transformations occur. This early extraction ensures that edge inscriptions, producer marks, and typographical details are preserved and later injected into downstream description prompts. Following transcription, the second stage performs background extraction and perspective rectification using rembg with U2-Net contour transforms to establish a standardized rectangular boundary while preserving the raw scan. To handle diverse photographic lighting and varying surface geometries, a shape- and scene-aware router dynamically selects among rembg, Otsu thresholding, and GrabCut algorithms based on background luminance and contour properties. When automated segmentation yields irregular boundaries, an interactive Crop Tuner allows conservators to adjust parameters, deskew rotated scans, and refine crops.

Quality assurance and descriptive generation occupy the third and fourth stages of the ingestion framework. To maintain archival integrity, stage three executes near-duplicate detection by comparing newly rectified scans against existing holdings using a dual-threshold filter. Scans exhibiting a CLIP cosine similarity exceeding 0.95 combined with a structural similarity index measure surpassing 0.9 are flagged as probable duplicates for curatorial review, preventing duplicate database entries without automatic deletions. Once verified, stage four directs the cropped label image and the stage-one transcription into gemma4:26b, an open-weights model of 25.8 billion parameters executed locally in Q4_K_M quantization under an Apache 2.0 license via Ollama. Prompted with visual and textual inputs, the language model generates a structured cataloguing description that systematically categorizes visual motifs, heraldic shields, stylistic characteristics, typography, and compositional hierarchies. This dual-input conditioning ensures that the resulting narrative accurately captures printed nomenclature alongside artistic themes, providing human-readable intermediate metadata that archivists can inspect and edit.

The final ingestion stage prepares generated descriptions for fast, hybrid natural-language search across distributed representations. Rather than indexing descriptions solely as monolithic blocks, WineEyes segments each record into non-empty line units and full texts, encoding each through the multilingual BGE-M3 model. This dual embedding generates both a 1024-dimensional semantic dense vector and a lexical sparse vector containing token-level weights, which are simultaneously ingested into the Qdrant vector engine. During live retrieval, Qdrant executes parallel independent searches across dense semantic embeddings and sparse lexical indices. Semantic dense retrieval captures conceptual associations—such as linking queries for heraldic crests to lions, crowns, and shields—while sparse lexical weights preserve exact matches for vintage dates, producer titles, and regional designations. Individual line-segment matches are subsequently collapsed to a single maximal rank per physical label before Reciprocal Rank Fusion combines the dense and sparse rankings with a smoothing parameter of k=60, storing stable identifiers in every vector payload.

The technical deployment of WineEyes relies on an integrated multi-container architecture orchestrated through Docker Compose to guarantee local execution and platform independence. A centralized FastAPI application coordinates communication between local inference engines, image transformation modules, indexing processes, and user-facing endpoints. Model serving is delegated to a local Ollama container handling optical character recognition and multimodal description generation, while Qdrant operates within a dedicated vector storage service. Relational data—including user accounts, role definitions, curated collections, institutional tags, and curator notes—is managed through MariaDB using the provided WineEyes.sql schema, which omits pre-populated user records and credentials. The authors note that the multi-container design, coordinating Ollama, Qdrant, and MariaDB via Docker Compose, exceeds the single-container limits of platforms like Code Ocean, prompting the release of configured environment definitions on GitHub under the MIT License. Local execution ensures that institutions maintain complete control over sensitive cultural collections without transmitting images across third-party cloud interfaces.

To evaluate retrieval efficacy, the researchers established an isolated pilot benchmark comprising 50 curated wine labels and 15 distinct natural-language queries. The query set was divided evenly across three operational categories: five textual queries targeting explicit strings such as bodegas and vintages, five simple iconographic queries specifying isolated visual motifs, and five relational iconographic queries combining multiple entities, interactions, and stylistic descriptors. To prevent benchmark contamination, the development team authored all 15 queries without viewing sample labels, cached transcriptions, or vision-language outputs, reserving the entire set exclusively for testing rather than prompt tuning. Ground truth was established by a domain specialist who performed binary relevance assessments over candidate pools generated via depth-k pooling across all evaluated systems. Additionally, the software suite incorporated an automated evaluation harness executing an audit of 355 verification checks with zero critical failures, validating canonical document identifiers, segmentation structures, metric calculations, and ranking completeness before final scoring.

Experimental results across the 15 evaluation queries revealed that the segmented hybrid indexing architecture achieved the strongest overall performance among all tested configurations. On macro-averaged point estimates, the segmented hybrid pipeline attained a normalized discounted cumulative gain at rank 10 (nDCG@10) of 0.727, a mean average precision at rank 10 (MAP@10) of 0.603, precision at rank 5 (P@5) of 0.547, precision at rank 10 (P@10) of 0.380, and a mean reciprocal rank at rank 10 (MRR@10) of 0.833. Full-description indexing yielded an nDCG@10 of 0.676 and a MAP@10 of 0.513. Historical baseline comparisons were restricted to depth-10 rankings, where BM25 fusion achieved an aggregate nDCG@10 of 0.708 (and top textual nDCG@10 of 0.987), while zero-shot CLIP achieved 0.530. Standalone dense optical character recognition scored 0.369, vision-language dense scored 0.630, and combined OCR-VLM dense reached 0.610. The segmented hybrid achieved leading scores on simple iconographic (0.705) and relational iconographic (0.539) queries.

To assess whether observed ranking differences were statistically meaningful given the modest sample size, the researchers performed paired significance testing with 10,000 bootstrap resamples. Comparing the segmented hybrid against the strongest historical baseline, BM25 fusion, yielded a mean difference of ΔnDCG@10 = +0.020 with a 95 percent confidence interval spanning from -0.017 to +0.059, accompanied by a two-sided Wilcoxon signed-rank p-value of 0.38 and a permutation p-value of 0.34. The corresponding MAP@10 difference was ΔMAP@10 = +0.008 with a confidence interval between -0.036 and +0.051 (Wilcoxon p=0.95). Comparing the segmented hybrid to full-description indexing produced an advantage of ΔnDCG@10 = +0.052 (95 percent CI [-0.080, +0.174], p=0.58). Because all confidence intervals crossed zero, the authors characterized the segmented hybrid as a promising configuration on this sample rather than an established superior architecture. Furthermore, ablation confirmed that dense-only (0.718 nDCG@10) and sparse-only (0.713 nDCG@10) branches underperformed hybrid fusion (0.731 nDCG@10), while varying Reciprocal Rank Fusion constants between 10 and 100 showed minimal fluctuation.

Component-level validation further illuminated the operational precision of the individual processing modules comprising the WineEyes framework. On the 50-label sample, automated cropping utilizing rembg and quadrilateral rectification successfully generated valid crops for 46 labels (92 percent), leaving four edge cases appropriately handled by the interactive Crop Tuner. Optical character recognition evaluated against manual transcription ground truth demonstrated a mean character error rate of 0.176, a word error rate of 0.278, and a token-level F1 score of 0.797. In a blind assessment of the 50 generated descriptions, gemma4:26b achieved an average textual fidelity score of 4.68 out of 5 under single-annotator review. Benchmarked on a workstation equipped with an Intel Core Ultra 9 275HX processor, 32 gigabytes of RAM, and an NVIDIA GeForce RTX 5090 Laptop GPU with 24 gigabytes VRAM, search latency across 15 queries averaged 199.4 milliseconds, with a median of 189.4 milliseconds and a 95th-percentile latency of 313.7 milliseconds, excluding a one-time 6.63-second model load.

Computational profiling indicated that one-time data ingestion represents the dominant processing expenditure of the WineEyes system, while search querying remains sub-second. On the RTX 5090 GPU, processing each label averaged 4.8 seconds for character recognition and 19.4 seconds for vision-language generation, totaling 24.2 seconds per label and completing the 50-label set in 20.2 minutes. Subsequent embedding and index generation required an additional 6.9 minutes (8.3 seconds per item). Extrapolating these measurements linearly indicates that ingesting an entire 25,000-label archive would require roughly 168 GPU-hours for description generation and approximately 225 GPU-hours for complete end-to-end indexing. While curatorial personnel at a Sherry-wine museum conducted informal in situ validation confirming the practical utility of query shortlists over manual searching, the authors emphasized several study constraints. These include reliance on a pilot-scale 50-label dataset, single-expert binary annotations without inter-rater reliability metrics, uncalibrated fusion scores, and evaluation limited to flat Spanish-language ephemera. Although architecturally adaptable to other flat heritage collections such as posters and postcards, cross-domain transfer remains unproven.

Subject of Research: Natural-language iconographic retrieval software for digitized heritage wine label collections

Article Title: WineEyes: Natural-language iconographic search for heritage wine labels

Article References: Mota, J. M., Baena-Pérez, R., Vargas-García, I., & Romo-Grilo, P. (2026). WineEyes: Natural-language iconographic search for heritage wine labels. SoftwareX, 36, Article 103080. https://doi.org/10.1016/j.softx.2026.103080

Image Credits: AI Generated

DOI: 10.1016/j.softx.2026.103080

Keywords: cultural heritage retrieval, wine label ephemera, caption-mediated search, vision-language models, optical character recognition, dense-sparse fusion, reciprocal rank fusion, vector databases, Qdrant, BGE-M3 embeddings, gemma4 vision model, glm-ocr, image rectification, near-duplicate detection, open-source software, local model inference, information retrieval evaluation, digital humanities

News Source: Denise Maddox. (November 12, 2026). Local Vision-Language System Retrieves Heritage Wine Labels Using Segmented Hybrid Search. Scienmag.

Tags: BGE-M3 embeddingscaption-mediated searchcultural heritage retrievaldense-sparse fusiondigital humanitiesgemma4 vision modelglm-ocrimage rectificationinformation retrieval evaluationlocal model inferencenear-duplicate detectionopen-source softwareoptical character recognitionQdrantreciprocal rank fusionvector databasesvision-language modelswine label ephemera
Share12Tweet7Share2ShareShareShare1

Related Posts

Partial Steel Ring Confinement Boosts Strength of Recycled Concrete Columns

Partial Steel Ring Confinement Boosts Strength of Recycled Concrete Columns

November 17, 2026
Mesoscale Aggregate Dynamics Govern Shear Transfer in Modeled Natural Aggregate Concrete

Mesoscale Aggregate Dynamics Govern Shear Transfer in Modeled Natural Aggregate Concrete

November 17, 2026

Optimal Grid Connection Capacity Balancing Costs in Hybrid Renewable Microgrids

November 17, 2026

Solvothermal Synthesis Yields Bismuth Selenide Nanoparticles with Distinct Phonon Modes

November 17, 2026

POPULAR NEWS

  • Chinese Visceral Adiposity Index Shows Nonlinear Link to Cardiovascular Disease Risk

    Chinese Visceral Adiposity Index Shows Nonlinear Link to Cardiovascular Disease Risk

    29 shares
    Share 12 Tweet 7
  • Review Links Skin Microbiota and T-Cell Crosstalk to Venous Disease Progression

    29 shares
    Share 12 Tweet 7
  • Pre-Curing Rheology Enables Electrospun Silicone Elastomer Coatings for Cochlear Implants

    29 shares
    Share 12 Tweet 7
  • Solvothermal Synthesis Yields Bismuth Selenide Nanoparticles with Distinct Phonon Modes

    29 shares
    Share 12 Tweet 7

About

We bring you the latest biotechnology news from best research centers and universities around the world. Check our website.

Follow us

Recent News

Female Fruit Flies Show Superior Survival Against Pathogenic Fungus Compared With Males

Nest Sanitation Practices Influence Gut Microbiota and Development in Starling Nestlings

Inhibiting MCP-1 Reduces BMP-2 and Crystal Adhesion in Injured Kidney Cells

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 88 other subscribers
  • Contact Us

Bioengineer.org © Copyright 2023 All Rights Reserved.

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
No Result
View All Result
  • Homepages
    • Home Page 1
    • Home Page 2
  • News
  • National
  • Business
  • Health
  • Lifestyle
  • Science

Bioengineer.org © Copyright 2023 All Rights Reserved.