Senior Big Data Infrastructure Engineer

Company: INSPYR Solutions
Location: San Francisco
Posted on: April 27, 2025

Job Description:

Senior Big Data Infrastructure Engineer - Long Term Project - San Francisco, CATitle: Senior Big Data Infrastructure Engineer

Candidates should take the time to read all the elements of this job advert carefully Please make your application promptly.
Location: San Francisco, CA
Duration: Long Term Project - 6+ Months
Compensation: $80-100/hr
Work Requirements: US Citizen, GC Holders or Authorized to Work in the U.S.

Summary
The Ad Infrastructure team is looking for a seasoned Software and Cloud Infrastructure Engineer to help build and scale our data processing platforms. This role is focused on enabling infrastructure and DevOps capabilities for high-scale batch and streaming workloads, including Spark and Flink on EKS. Our mission is to reduce the cognitive load on application and data teams by delivering scalable, secure, and easy-to-operate infrastructure and tools that accelerate development and delivery.

What You'll Do Design, build, and maintain infrastructure automation and tooling that enables developers to deploy and operate cloud-native applications.
Implement best practices for secure, scalable, and cost-efficient cloud operations across AWS.
Contribute to and maintain a robust library of shared Terraform modules and infrastructure-as-code standards.
Participate in root cause analyses (RCAs) and postmortems to continuously improve reliability and operational practices.
Support container orchestration and workload scaling on Kubernetes (EKS) for Spark and Flink jobs.
Collaborate with development teams to ensure seamless onboarding and operational support for infrastructure tooling and platforms.
Participate in on-call rotations and own uptime and reliability of key systems.
Drive adoption of DevOps practices, including CI/CD workflows, observability, and incident management.

What You Bring 7+ years of experience in infrastructure engineering, DevOps, or cloud-native softwareengineering.
4+ years of hands-on experience with Kubernetes, preferably on Amazon EKS.
3+ years of experience with Python or Go for infrastructure automation.
3+ years working with data processing frameworks such as Apache Airflow, Spark, or Flink.
2+ years designing and operating Databricks environments at scale on AWS, including performance tuning and securing workloads.
Expertise in AWS services (EC2, S3, IAM, VPC, etc.), infrastructure automation, and Terraform.
Experience with container technologies (Docker) and CI/CD tools like Spinnaker.
Strong communication skills and a collaborative mindset for working across engineering teams.

Nice to Have AWS Certifications (e.g., Solutions Architect, DevOps Engineer).
Experience with Unix/Linux systems and shell scripting.
Familiarity with all three: Airflow, Spark, and Flink.
Experience operating service mesh technologies like Istio.
Experience running Spark or Flink workloads on Databricks.

Our benefits package includes: Comprehensive medical benefits
Competitive pay
401(k) retirement plan
---and much more!

About INSPYR Solutions
Technology is our focus and quality is our commitment. As a national expert in delivering flexible technology and talent solutions, we strategically align industry and technical expertise with our clients' business objectives and cultural needs. Our solutions are tailored to each client and include a wide variety of professional services, project, and talent solutions. By always striving for excellence and focusing on the human aspect of our business, we work seamlessly with our talent and clients to match the right solutions to the right opportunities. Learn more about us at inspyrsolutions.com.
INSPYR Solutions provides Equal Employment Opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, or genetics. In addition to federal law requirements, INSPYR Solutions complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities

Keywords: INSPYR Solutions, Santa Cruz , Senior Big Data Infrastructure Engineer, Engineering , San Francisco, California