Data Engineer / Analyst (Python - Web Scrapping)

Install, operate and maintain database infrastructure. • Manage backups and security issues. • Collaborate with data scientists and analysts to understand data requirements and design efficient data models and architectures. • Collaborate with cross-functional teams to identify and implement data infrastructure and tooling improvements. • Develop, test and maintain crawlers / scrapers for various websites or REST APIs when available • Design and test strategies to optimize calls to a website and to ensure the quality of the extracted data • Push scraped data into relational databases • Identify additional relevant sources to scrap • Stay updated with emerging technologies, industry trends, and best practices in data engineering. • Perform data analysis and troubleshooting to resolve data-related issues and provide timely resolutions. • Contribute to prioritization decisions (feasibility/quality vs. effort) • Build connection with backend Requirements • You have 2+ years of experience as Data Engineer using Python • Proficiency in working with relational databases (SQL) and NoSQL and AWS • You have experience with a web scraping framework like Scrapy, and can understand which framework is more suitable in each situation • You like to code following best practices (e.g., clean & simple code, continuous testing, version control) • Knowledge of data modeling and database design principles • Ideally, you are familiar with cloud-based environments and can automate the deployment of scrapers on a cloud architecture • Strong understanding of data warehousing concepts and data pipeline architectures. • Proficiency in data manipulation, transformation, and aggregation techniques. • Clean and structure the collected data, ensuring its accuracy, consistency, and organization for further analysis and processing. • Excellent problem-solving and analytical skills. • Strong attention to detail and the ability to work with large and complex data sets. • You like working in a small team setting • You are not afraid of challenges, are proactive and self-motivated