The customer is a Data Platform that offers a unified environment for ingesting, transforming, analyzing, and storing data, with deployment options across cloud, on-premises, and hybrid or multi-cloud infrastructures.
We are seeking a Senior Java Engineer to support a collection of 90+ metadata ingestion connectors (databases, BI tools, etc.), driving bug fixes, refactoring, and rebuilds as necessary to ensure reliable catalog integration.
Requirements:
8+ years of experience in Java development.
Experience building integrations across a variety of protocols, including REST, JDBC, GraphQL, SOAP, and direct FileSystem (e.g., HDFS, S3) interactions.
Experience within a metadata extraction architecture where a lightweight, on-premise scanner agent runs plugins to perform the data source connection and push metadata to a central cloud platform.
Proficiency with modern development tools, specifically IntelliJ, GitHub for version control, and Jira for workflow management.
Experience deploying and debugging applications on both Linux and Windows servers, as well as scheduling jobs via cron or command-line interfaces.
At least Intermediate English proficiency.
Responsibilities:
Build new connector plugins using a Java-based SDK to extract and map source-specific metadata (schemas, keys, data models) into the platform.
Migrate existing connectors from Scala to Java, maintaining functionality while introducing enhanced capabilities supported by the new SDK.
Design and implement logic to capture complete end-to-end data lineage by parsing SQL (Spark SQL, T-SQL, etc.), Python, and AWS Glue scripts.
Extend connectors with advanced features (e.g., data sampling, fingerprinting, filtering) and serve as Layer 2 (L2) support for troubleshooting connector-related issues.
