Location: Belgrade, Serbia
About the Role:
We are building the next-generation data platform behind large language model development. Our initial training corpus spans 2–5 TB of raw, unstructured text across web, legal, medical, technical, and literary sources. Because the engineering and quality of this corpus is the single biggest determinant of final model quality, you will own the distributed platform that produces it.
What You'll Do:
- Design Lakehouse Architecture: Build and optimize end-to-end data pipelines using Apache Iceberg to transition raw, multi-terabyte unstructured text into clean, structured, and training-ready datasets.
- Scale Text Processing: Build and maintain high-throughput distributed pipelines in Ray to handle heavy document extraction, cleaning, and tokenization.
- Build Deduplication Systems: Implement exact and near-duplicate text detection (MinHash/LSH, suffix arrays) operating efficiently at web-scale.
- Implement Quality Filtering: Orchestrate heuristic and model-based classifiers, perplexity filtering, and high-volume data enrichment pipelines.
- Prevent GPU Starvation: Optimize data delivery I/O paths and caching layers to ensure our high-performance GPU clusters remain fully saturated during embedding generation and training.
- Own Data Lineage & Validation: Create reproducible, auditable dataset snapshots linked to experiment tracking, and implement strict data validation checks to prevent silent data corruption in training pipelines.
- Set Engineering Standards: Lead code reviews, implement robust automated testing for data transformations, and mentor mid-level engineers on the team.
Required:
- 6+ years in data engineering, with clear experience designing, scaling, and maintaining distributed production pipelines.
- Production experience with distributed computing paradigms (Ray, Dask, or PySpark/Spark). You must know how to execute massive cluster-scale jobs, handle data skews, and manage distributed memory/I/O bottlenecks. (We lean heavily on Ray for our core pipelines, but deep expertise in Spark/Dask translates well).
- Strong Python engineering skills, solid SQL, and comfortable dealing with multi-node infrastructure.
- Hands-on experience with modern table formats (Apache Iceberg or Delta Lake) and cloud object storage.
- Experience with workflow orchestration at scale (Argo Workflows, Airflow, or Dagster).
- The architectural judgment to know when a massive distributed job is bottlenecked on cluster shuffle, network I/O, or an inefficient regex.
- Great English language skills.
Strongly Preferred:
- Prior experience processing LLM pre-training corpora or massive web-scale text (e.g., Common Crawl derivatives, quality classifiers).
- Deep familiarity with high-performance, single-node processing libraries (Polars, DuckDB) and the Apache Arrow memory model to optimize worker-level processing.
- Experience parsing complex, multi-modal document structures (PDFs, HTML, tables) using frameworks like
- Unstructured.io or similar data connectors.
- Experience implementing distributed caching layers (JuiceFS, Alluxio, or similar) to optimize data throughput for deep learning workloads.
- Familiarity interacting with high-throughput inference serving frameworks (Triton Inference Server, vLLM) to scale pipeline embedding generation.
- Familiarity with text deduplication algorithms (MinHash/LSH, SimHash, suffix arrays).
- Experience with ML tracking and data lineage tools (MLflow, W&B, or DVC).
Nice to Have:
- Experience implementing strict programmatic data validation (e.g., Pandera, Great Expectations).
Familiarity with deep-learning specific data formats (safetensors, WebDataset) and high-speed Rust-based tokenizers.
- Familiarity with REST catalogs and data governance frameworks (Apache Polaris, DataHub) for managing Lakehouse access.
- Open-source contributions to Ray, Spark, Iceberg, or ML infrastructure tooling.
- Experience working with distributed vector databases (Milvus, Qdrant, etc.) as downstream search indices.
- Familiarity with Kubernetes infrastructure components (KubeRay, Docker, isolated node pools).
Why This Role?
- Direct Impact: The pipeline you build will be the reason the model succeeds or fails. Few data engineering roles have that direct a line to the core AI product.
- Real Resources: You will have direct access to cutting-edge local hardware and a dedicated training schedule — this is not a queue you wait in.
- Cross-Functional Collaboration: You will work side-by-side with our Machine Learning Engineers, NLP researchers, and linguists to define what world-class data quality looks like.
Peer Support: You will operate in a peer-level partnership with a second Senior Data Engineer owning streaming and storage infrastructure. This is not a one-person heroics.
What we offer:
- Challenging projects in private sector.
- Sustainable and socially beneficial projects in public sector.
- Working with well-educated and experienced colleagues.
- Paid certification and education in country and abroad(training, courses, exams etc.).
- Pleasant and friendly office environment.
- Flexible working hours.
- Parking.
- Fruit, snacks and drinks.
- Team buildings and Company celebrations(Secret Santa, New Year gifts for kids, Women’s Day).
- Private health care for all employees.
- Paid prenatal test for pregnancy.
- 100% paid sick leave.
- Paid Gym & Fitness.
- For the birth of a child, the employee receives 500 EUR bonus.
Selection process:
- We will contact only short-listed candidates.
- We optimised the selection process. The first interview will be with HR, followed by the technical interview. If the interview goes well, you may be invited to another technical interview or you will receive a job offer.
