Generative AI Application Engineer

Generative AI Application Engineer
Location: United States (Remote)
Employment Type: Full Time (W2)
Working hours: Must be able to work remotely in the US on East Coast time zone. Work day starts at 8:30am ET - Exceptions for PT candidates for 9am ET / 6am PT and when needed, meetings at 8:30am ET.
Industry: Financial Technology / Data Analytics SaaS
Reports To: AI/Modeling Team Leadership

About This Opportunity
Purple Squirrel Enterprises has been supporting startups and global firms through growth, change, and expansion for 19+ years. We provide flexible HR and recruiting solutions that allow leaders to focus on strategy while we handle people operations — and this search is no exception.

We are exclusively engaged by a well-established SaaS analytics company serving the banking industry to help them identify and hire a hands-on Generative AI Application Engineer. We are embedded directly with their leadership team, have deep knowledge of the company, the culture, and what it takes to succeed here — and we will be your dedicated recruiting partner from first conversation through offer.

This is not a blind submission. When you apply, you are working with a team that knows this opportunity inside and out and will advocate for you every step of the way.

💜 If this opportunity resonates with you, we encourage you to apply. We will be in touch promptly.
Applications submitted directly to our client will be redirected back to Purple Squirrel Enterprises. To be considered for this role, all candidates must apply through this posting.

About Our Client
Our client is a well-known and respected provider of SaaS-based data analytics to the banking industry. Their platform helps banks and credit unions turn raw operational and customer data into the insight they need to run the business today — while guiding them through the next phase of their data and analytics journey.

Their internal Modeling and AI team leads both product and internal efforts in advanced analytics, and they are actively expanding that team's ability to ship production generative AI features directly into the platform their banking clients rely on every day.

This is a data-sensitive, regulated environment. The work is real, the data is real, and the standard for what counts as "in production" is high.

About the Role
Our client is looking for a Generative AI Application Engineer to design, build, and ship production generative AI features — API-based services, agents, and tools built directly against large language model (LLM) APIs — along with the application and data layers those features depend on.

This is a hands-on, mid-level engineering role, not a research position and not primarily a machine learning modeling position. The emphasis is on software engineering depth combined with applied generative AI experience: substantial Python and API development history, direct development at the LLM API level, and strong SQL skills.

Development is performed using an AI-First / Specification-Driven development process. You'll work closely with product management, data engineering, and other members of the internal AI/Modeling team to translate requirements into shipped features — and to find opportunities to increase organizational efficiency using generative AI, models, and heuristics along the way.

Candidates must have generative AI applications running in production today and must be able to name them, describe their own contribution, and provide reachable references who can confirm that contribution. Machine learning experience is valued and will be used, but it is not the basis on which this role is filled.

What You'll Do
Generative AI Application Development

• Design, build, and ship production generative AI features developed directly against LLM APIs — prompt construction, structured output handling, tool/function calling, retrieval, context management, and failure handling
• Build and maintain the evaluation and regression harnesses that determine whether a generative AI feature is behaving correctly as models, prompts, and data change
• Integrate generative AI components with existing product services, APIs, and data models
• Work with models hosted on Hugging Face and with internally hosted inference engines, including configuration and deployment of the serving layer

Software Engineering and System Design

• Design, build, and maintain API-based applications and services following the team's software and product life-cycle processes
• Write and tune SQL against relational data models for application state, feature data, evaluation datasets, and product analytics
• Deploy and operate application components across multiple resource types in a hyperscaler environment (VMs, containers, object storage, RDBMS services) — primarily Microsoft Azure
• Contribute to analytics product work that doesn't involve AI/models, when needed — this is not the primary focus, but it happens

Machine Learning and Model-Based Analytics

• Contribute to model-based analytics such as customer segmentation, churn forecasting, and lifetime value forecasting, working alongside senior members of the AI/Modeling team
• Support internal initiatives that use generative AI, models, and heuristics to increase organizational efficiency

Collaboration and Cross-Functional Delivery

• Work with product management, data engineering, and AI/Modeling teammates to translate requirements into shipped features
• Participate in code review — both giving and receiving
• Communicate progress, technical trade-offs, and blockers clearly to your team and to stakeholders outside it

Quality, Testing, and Operations

• Write automated tests appropriate to the work, including evaluation tests for generative AI components and model-evaluation tests for ML components
• Monitor deployed AI applications in production for accuracy, latency, token consumption, and cost; participate in production support as needed
• Follow security, data handling, and change-management practices required when working with sensitive financial data

What This Role Is Not
This is not a research role, and it is not a machine-learning-modeling-first role. It is not a "consume an API endpoint and call it AI" role, and it is not a no-code/low-code tooling role. If your generative AI experience is limited to assembling existing tools or building weekend prototypes, this isn't the right fit. If you've shipped real generative AI applications to production, can name them, and can point to someone who'll confirm your role in building them — this is exactly the right fit.

What Success Looks Like

• Shipping, Not Prototyping: You've moved generative AI features from idea to production, with evaluation and monitoring in place, not just a demo
• Trusted Technical Judgment: Product management and data engineering count on you to flag what's feasible, what's risky, and what needs more time
• Production Ownership: You know how your features are performing in the wild — accuracy, latency, cost — and you act on that information
• Team Contribution: Code reviews, cross-team communication, and collaboration with the AI/Modeling team reflect someone who makes the whole team better, not just their own output

What We're Looking For
Required

• At least 5 years of professional software development experience, ideally within a software product organization, including hands-on development of API-based applications and services deployed and used in production
• At least 4 years of hands-on Python development experience — deep enough to read, explain, and defend your own design decisions
• At least 2 years of substantive generative AI development at the LLM API level — prompt design, structured outputs, tool/function calling, retrieval, and evaluation. Assembling no-code/low-code tooling does not satisfy this requirement
• At least two non-trivial generative AI applications you helped build that are in production today, with the ability to describe the problem, architecture, your specific contribution, and how it's evaluated/monitored
• Strong SQL skills — joins, aggregation, window functions, CTEs — plus a working understanding of RDBMS performance and role-based access controls
• At least 4 years of experience working across multiple resource types in a hyperscaler environment (VMs, containers, object storage, RDBMS services). Microsoft Azure preferred; equivalent AWS or GCP experience considered
• Proficiency with version control practices and tools (Git and/or Azure DevOps)
• Reachable professional references covering the experience above, including your production generative AI work (see Reference Requirements below)

Highly Desirable

• Hands-on experience with models on Hugging Face — selection, serving format/quantization choices, and evaluation
• Direct, personal involvement deploying models for self-hosted LLM inference (LightLLM, vLLM, SGLang, TensorRT-LLM, TGI, Ray Serve, Triton, Ollama, llama.cpp, or similar). Gateway/routing layers like LiteLLM also relevant
• Self-hosted inference deployments supporting production applications (not just experiments), including capacity planning, batching/concurrency, versioning, and monitoring
• Experience building applications/UIs that interface with self-hosted, open-source LLMs via APIs

Nice to Have

• Experience with Amazon Bedrock or Microsoft Foundry (formerly Azure AI Foundry)
• Machine learning experience — feature engineering, model training/evaluation using Pandas, scikit-learn, or similar
• Python charting libraries (Matplotlib, Seaborn, Plotly)
• Azure SQL and PySpark experience for large datasets
• Additional experience in C#, Java, or Scala
• React/TypeScript UI development (AI Elements or shadcn/ui a plus)
• Exposure to banking, financial services, or another regulated data environment

Reference Requirements
This role carries a specific reference expectation, stated up front so candidates can plan for it:

• Reachable professional references are required covering the experience claimed in this posting
• At least one reference must come from each organization where your production generative AI applications were built and deployed
• References must be able to speak directly to what you personally built, which technical decisions you owned, and how the application performed in production
• References will be contacted, and an offer is contingent on verification of the production generative AI experience described above
• If a former employer's policy or an NDA limits what a reference can confirm, raise it during your first interview — not at the reference stage. We'll work out an acceptable alternative

Compensation & Benefits

• Base salary: $135,000 – $150,000, based on current market data for this role. Candidates who clearly exceed the core requirements (verified production GenAI applications + strong references) may be considered outside this range
• Full-time, W2 position, remote-friendly work environment
• $1,100/month company benefit contribution toward eligible pre-tax benefits (medical, dental, vision, HSA) — flexible to allocate based on your plan selection
• Medical, dental, and vision coverage, including HSA-compatible plan options
• UHC Rewards program — earn up to $300 (non-HSA) or $1,000 (HSA) annually through wellness activities
• Employer-paid Life/AD&D insurance (1x salary up to $200,000)
• Employer-paid Short-Term and Long-Term Disability coverage
• Voluntary benefits available: additional Life/AD&D, Critical Illness, Accident, and Hospital Indemnity coverage
• Employee Assistance Program (EAP)
• 401(k) plan with employer match — up to 4.5% total match for employees contributing 6% of salary, immediately vested
• Company holidays plus a Flexible Time Off (FTO) policy — no fixed PTO cap, built on mutual trust and manager coordination

Working With Purple Squirrel Enterprises
We are exclusively engaged by this client — meaning we are not one of many agencies submitting resumes. We are their dedicated recruiting partner, embedded in their process, and we know this role inside and out.

Here's what that means for you as a candidate:

• A dedicated PSE point of contact throughout the entire process
• Advocacy with the hiring team to make sure the role is mutually beneficial
• Preparation for each interview stage so you can put your best foot forward
• Honest, timely feedback at every step
• Support through offer negotiation to ensure the outcome is right for you and the company
• Deep experience navigating complex hiring processes to get great candidates across the finish line

Our client is an equal opportunity employer committed to building a diverse and inclusive team. They welcome applicants of all backgrounds, experiences, and perspectives.

Originally posted on Himalayas