Big Data Platform Engineer
Quick facts
- Pay
- $71/hour
- Type
- Full-time
- Location
- Chicago, IL
- Skills
- Python, Linux
- Industry
- Information Technology — 1359 open jobs
- Visa in listing
- H-1B, OPT
- Apply
- Free to apply — never pay anyone for a job offer or visa sponsorship
Job description
Job ID: J54892
Job Title: Big Data Platform Engineer
Location: Chicago, IL
Duration: 8 Months + Extension
Domain Exposure: Healthcare
Client: To Be Discussed Later
Employment Type: W-2, 1099, C2C
Work Authorization: US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other
Job Title: Big Data Platform Engineer
Location: Chicago, IL
Duration: 8 months
Bill Rate: $71/hour (C2C)
Job Type: W-2, 1099, C2C
Client: To Be Discussed Later
Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other
Job Description: Seeking an experienced Spark, HDFS, and Kafka Administrator/Engineer to manage, support, and optimize enterprise-grade big data platforms. Deep hands-on expertise with Apache Spark, HDFS, and Kafka is required and will be the primary focus of this role. Experience administering these technologies within a Cloudera environment is preferred but secondary to strong expertise in the core underlying platforms.
Key Responsibilities:
- Spark Administration: Deploy, configure, monitor, tune, and troubleshoot Apache Spark environments and workloads. Support Spark job performance, resource management, capacity planning, and production operations.
- HDFS Administration: Manage and support HDFS storage, including capacity, performance, availability, permissions, replication, and troubleshooting. Ensure robust integration between distributed compute platforms and storage.
- Kafka Administration: Deploy, configure, manage, monitor, and troubleshoot Apache Kafka clusters, topics, producers, and consumers. Support performance, capacity, availability, security, and production operations.
- Monitoring & Performance Tuning: Monitor platform connectivity, health, capacity, and performance using tools such as Grafana, Splunk, and related management utilities. Proactively identify and resolve bottlenecks across Spark, HDFS, and Kafka.
- Security & Compliance: Implement and manage security configurations, including Kerberos, SSL/TLS, authorization policies, HDFS ACLs, and enterprise IAM/LDAP integrations.
- Automation & Scripting: Develop and maintain automation using Python, Bash, or Perl for routine administration, configuration management, monitoring, and operational support.
- Troubleshooting & Support: Provide expert-level troubleshooting and technical support across the big data tech stack, including participation in an on-call rotation.
- Platform Lifecycle: Support installations, patching, upgrades, migrations, and maintenance activities while minimizing downtime. Support Cloudera Hadoop/CDP administration as needed.
Qualifications & Requirements:
- Core Experience: Minimum 3–5 years of hands-on production administration, configuration, performance tuning, and sizing experience with Apache Spark, HDFS, and Apache Kafka.
- Linux Administration: Solid hands-on experience with Linux operating systems, logging in, navigating, and executing command-line administration.
- Monitoring & Observability: Practical experience using enterprise monitoring and operational tooling such as Grafana and Splunk.
- Security Foundations: Solid understanding of Kerberos authentication, SSL/TLS encryption, IAM/LDAP integration, and general security concepts for distributed data architectures.
- Communication & Collaboration: Excellent analytical, problem-solving, and written/verbal communication skills.
- Education: Bachelor’s degree in Computer Science, Information Systems, or a related field.
Preferred / Desirable: Experience with Cloudera Hadoop/CDP and Cloudera Manager (preferred, but secondary to foundational Spark, HDFS, and Kafka expertise). Cloudera certifications or equivalent big data
Equal Opportunity Employer We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.
Visa sponsorship record: Robotics Technologies LLC
Sponsors actively| Fiscal year | H-1B LCAs | USCIS approvals | USCIS denials | PERM |
|---|---|---|---|---|
| FY2026 | 86 | 30 | 5 | — |
| FY2025 | 82 | 8 | 0 | — |
| FY2024 | 36 | 11 | 0 | — |
✔ Robotics Technologies LLC filed 45 H-1B applications for this kind of role (Software Developers) in the period above.
Also filed: H-1B1 Singapore × 1 — a good sign for Australian (E-3), Chilean and Singaporean (H-1B1) candidates.
Roles they sponsor most
- data analyst22 LCAs · median $90k
- software developer13 LCAs · median $84k
- software engineer10 LCAs · median $84k
- business analyst8 LCAs · median $80k
- data engineer7 LCAs · median $98k
Which visas can work for this job
Occupation: Software Developers (SOC 15-1252).
- Mentioned in the listing
H-1B, OPT
- H-1B cap-exempt employer
Regular cap-subject employer — a new H-1B needs to win the lottery in March (unless you already hold cap-counted H-1B status).
- TN (citizens of Canada and Mexico)
This kind of role may fit the USMCA profession “Computer Systems Analyst (or Engineer, for engineering-degree holders)” — it depends on the actual duties. No lottery, no cap; you need the matching degree or license.
- E-3 (Australia) and H-1B1 (Chile, Singapore)
This employer has filed before (H-1B1 Singapore: 1). Same degree requirement as H-1B, but no lottery and a separate quota that is rarely filled.
- O-1 (extraordinary ability)
For candidates with awards, publications, press, a high salary or critical roles at distinguished organizations. No cap, no lottery; the employer files a petition.
- STEM OPT extension (F-1 students)
Requires an E-Verify employer. We didn't find this company in the E-Verify list — ask HR.
Salary vs prevailing wage
Level III (experienced)Software Developers · Chicago-Naperville-Elgin, IL-IN. Annual prevailing wages set by the Department of Labor (OFLC).
| Level | Prevailing wage | H-1B lottery odds* |
|---|---|---|
| Level I | $92,789 | ~15% |
| Level II | $117,083 | ~31% |
| Level III | $141,398 | ~46% |
| Level IV | $165,693 | ~61% |
This job pays $147,680 a year (converted from an hourly rate, 2,080 hours) — that's Level III (experienced). In the wage-weighted H-1B lottery a Level III registration gets 3 entries; estimated selection chance about 46% — better than average.
* Odds are DHS projections for the FY2027 wage-weighted lottery (actual results vary by year and employer). Since the FY2027 cap season the H-1B lottery is weighted by wage level: Level I = 1 entry, II = 2, III = 3, IV = 4. The level is set by the offered wage against the prevailing wage for the occupation and worksite. Separately, a $100,000 fee for new H-1B petitions for workers outside the US was announced in 2025; as of September 2026 a federal court ruling keeps it unenforceable while appeals continue — check the current status.
Sources: U.S. Department of Labor OFLC disclosure data (H-1B/H-1B1/E-3 LCA, PERM), OFLC prevailing wage data, USCIS H-1B Employer Data Hub, E-Verify participating employers. Data loaded: LCA FY2024–FY2026, PERM, USCIS Data Hub, OFLC wages; updated 2026-10-03. Employers are matched by name, so records of companies with similar names can occasionally be mixed up. This is general information, not legal advice — talk to an immigration attorney about your case.