SISTRIX GmbH
Bonn
Software Craftsman & DevOp - SEO
Software Craftsman | DevOp
September 2016 - January 2018
1 year 5 months
full-time
Project
Bonn
🎯
Overview
As Software Craftsman & DevOp at SISTRIX, I built Big Data pipelines for SEO analysis with "You Build It, You Run It" methodology. CI/CD pipeline with Jenkins/Docker, data extraction with Spark/Hadoop for 450M+ keywords worldwide, and PaaS architecture with Apache Mesos/Marathon.
HTML parser with XPath for effective structuring of millions of keywords per country (Germany: ~100M, USA: ~52M), HTML crawler for 200M seeds, and automated acceptance tests for SaaS tools. Status dashboard with Play Framework for project transparency and operational overview.
⚡
Activities
- Big Data Pipeline Development: HTML parser with XPath for millions of keywords per country, API integration
- Data Extraction: HTML crawler with Spark/Hadoop for 200M seeds, structured data extraction
- DevOps & CI/CD: "You Build It, You Run It" pipeline with Jenkins/Docker, automated deployments
- PaaS Architecture: Apache Mesos/Marathon, AWS Route 53, scalable infrastructure
- Quality Assurance: Automated acceptance tests for SaaS tools, Cucumber testing
- Monitoring & Operations: Status dashboard with Play Framework, operational transparency
🔄
Methodology
- "You Build It, You Run It": DevOps Culture, End-to-End Ownership
- Big Data Processing: Spark/Hadoop, Scalable Data Pipelines
- CI/CD: Automated Testing, Continuous Deployment
- PaaS Architecture: Container Orchestration, Service Mesh
Technology Stack
Technologies and tools used in this project
⚙️
Backend
4
Java
Scala
Play Framework
ArangoDB
📦
Other
6
Akka
Data Storage
XPath
Cucumber
Testing
Quality Assurance
📊
Data & AI
5
Spark
Hadoop
Big Data Processing
Container Orchestration
MySQL
🚀
DevOps
5
Docker
Jenkins
CI/CD Pipeline
Apache Mesos
Marathon