Job Overview
We are hiring a Mid-Level Data Engineer to design and build OLTP (Online Transaction Processing) systems and pipelines. This role focuses on transactional system design, API-driven data flows, and building reliable, high-performance data services. The ideal candidate thinks in terms of system architecture, transactional integrity, and low-latency data handling rather than scheduled data warehouse loads.
Key Responsibilities
- Design, build, and maintain OLTP pipelines for real-time or near-real-time transactional data processing.
- Build and consume APIs to handle data ingestion, updates, and retrieval from operational systems.
- Design SQL Server database schemas optimized for transactional workloads — normalization, indexing, concurrency control, and locking behavior.
- Write and optimize SQL queries, stored procedures, and transaction logic for high-throughput, low-latency operations.
- Ensure data consistency, atomicity, and integrity (ACID compliance) across services.
- Develop ETL processes as needed to support data migration, system integration, or occasional batch sync between transactional systems.
- Handle API request/response cycles: validation, error handling, retries, and rate limiting on data-write operations.
- Monitor system performance, diagnose issues (deadlocks, query latency, contention), and tune for scale.
- Deploy and manage data services using Azure (Azure SQL Database, API Management, Azure Blob).
- Use SSIS (SQL Server Integration Services) for occasional data migration or integration tasks.
- Collaborate with backend engineers to integrate data layers into application services.
- Implement validation, idempotency checks, and integrity constraints at the transactional level.
- Create and maintain documentation for system architecture, API contracts, and database schemas.
Requirements
- Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent practical experience).
- 7–10 years of experience in SQL Server and relational database design for OLTP systems — normalization, indexing, transactions, isolation levels.
- Solid experience building or consuming APIs (REST) for data read/write operations.
- Working knowledge of Azure services: Azure SQL Database, API Management,, Azure Blob Storage.
- Experience working with transactional data in high-concurrency, production environments.
- Experience writing and optimizing SQL stored procedures for transactional systems.
- Hands-on experience with SSIS or similar tools for data migration/integration tasks.
- Working knowledge of ETL development for occasional batch sync or migration needs.
- Understanding of system design fundamentals: concurrency, caching, request handling, and service scalability.
- Experience collaborating with backend/application engineers to integrate data layers into services.
- Strong analytical, troubleshooting, and communication skills.