Only AI Jobs


AI Agent Experience Designer

ID: 9417

Type: Full-time

Category: Others

Company Name: Allied Benefit Systems, LLC

Location: Illinois (USA)

Education Level: Senior (5-10 years)

Visit company vacancy
Job Description

POSITION SUMMARY 

The Prompt Engineer will be responsible for optimizing, evaluating, and advancing the performance of AI-driven agents, such as chatbots and autonomous workflows to ensure they deliver accurate, efficient, and high-quality outcomes. This role combines analytical thinking, user experience insight, and technical expertise to monitor agent behavior, identify improvement opportunities, and implement enhancements that align with business goals.

ESSENTIAL FUNCTIONS 

  • Define and document required AI agent behavior, including edge cases, compliance constraints, escalation expectations, and response quality targets.
  • Convert benefits workflows and policy rules into implementable use cases, key data fields to capture, and testable scenarios that can be built and verified.
  • Maintain a clear mapping from requests to delivered behavior so gaps are visible and reviewable.
  • Drive stakeholder reviews and acceptance of behavior, flows, and decision logic before changes ship.
  • Craft and iterate prompt instructions and response rules that produce accurate, consistent answers in a regulated benefits context.
  • Build reusable prompt patterns and maintain a controlled prompt library with clear version history.
  • Review real interactions to identify failure modes, then adjust prompts, context, and fallbacks to improve outcomes.
  • Partner with engineering and QA to define regression test coverage and release criteria for prompt and flow changes.
  • Define the agent’s voice and interaction style so it is professional, on-brand, and easy for members and providers to follow.
  • Design multi-step conversation paths that handle ambiguity, corrections, and interruptions without losing the user.
  • Define when the agent should answer, when it should ask clarifying questions, and when it should hand off to a live person.
  • Validate that wording, terminology, and risk posture match benefits operations expectations.
  • Define how the agent should use plan documents to answer questions, including what to do when source material is missing or ambiguous.
  • Partner with contact center and workforce management teams to align the agent experience with SOPs and ensure workflows stay current as procedures change.
  • Partner with engineering on query shaping and grounding so document-based answers stay tied to supporting text.
  • Own the gold standard definition of intended agent behavior after launch, so teams can support and improve it without relying on vendor or institutional knowledge.
  • Maintain the core operating artifacts used day to day: use case catalog, flow maps, escalation rules, prompt library, and a running list of known failure modes.
  • Create lightweight training and working sessions for internal support partners and business SMEs.
  • Establish a change process for updating prompts and flows, including testing expectations, approvals, and version history.
  • Requirements and functional specifications for prioritized conversational AI use cases.
  • Use case catalog and key information requirements mapped to approved scenarios.
  • Conversation flow designs that include fallbacks, error handling, and escalation.
  • Versioned prompt library with testing notes and a clear change history.
  • UAT scripts and acceptance criteria, plus defect tracking through resolution.
  • Maintain a curated set of golden records and test cases that represent critical scenarios and edge cases and use them to support automated regression and quality testing.
  • Operator documentation that enables internal support and ongoing improvement.
  • Post-launch performance monitoring, including top failure modes and improvement plans.
  • Other duties as assigned

EDUCATION

  • Bachelor’s degree in Computer Science, Math, Statistics, UI/UX, Process Design or equivalent work experience required.  

EXPERIENCE AND SKILLS

  • 6+ years of experience across business analysis, conversational AI, UX writing, and related product roles. 
  • Direct experience designing and improving chatbots, voicebots, IVR modernization, or contact center automation.
  • Ability to translate business workflows into clear user requests, key information to capture (e.g., member identifiers, dates, plan options), and end-to-end conversation paths that engineering can build and QA can test.
  • Strong prompt design skills and a test-and-learn mindset using real interaction data.
  • Strong writing and editing skills with comfort in a regulated, customer-facing environment.
  • Comfort operating with ambiguity in a new and evolving role, and able to create processes and standards where none exist.
  • Ability to collaborate with cross-functional stakeholders and external partners.
  • Shipped conversational AI experiences in voice and/or contact center environments.
  • Experience working with plan documents, knowledge bases, and policy text in regulated environments. Able to translate source material into grounded answers and guardrails.

POSITION COMPETENCIES

  • Accountability
  • Analytical Problem Solving
  • Collaboration
  • Execution and Delivery
  • Quality and Risk Management
  • Systems Thinking
  • Technical/Functional Expertise

PHYSICAL DEMANDS

  • This is a standard desk role requiring extended sitting and computer work

WORK ENVIRONMENT

Remote

 

Here at Allied, we believe that great talent can thrive from anywhere. Our remote friendly culture offers flexibility and the comfort of working from home; while also ensuring you are set up for success. To support a smooth and efficient remote work experience, the internet connection must be obtained through a cable broadband or fiber optic internet service provider with speeds of at least 100Mbps download/25Mbps upload. Reliable internet service is essential for staying connected and productive.

The company has reviewed this job description to ensure that essential functions and basic duties have been included. It is not intended to be construed as an exhaustive list of all functions, responsibilities, skills, and abilities. Additional functions and requirements may be assigned by supervisors as deemed appropriate.

Compensation is not limited to base salary.  Allied values our Total Rewards, and offers a competitive Benefit Package including, but not limited to, Medical, Dental, Vision, Life and Disability Insurance, Generous Paid Time Off, Tuition Reimbursement, EAP, and a Technology Stipend.

Allied reserves the right to amend, change, alter, and revise, pay ranges and benefits offerings at any time.  All applicants acknowledge that by applying to the position you understand that the specific pay range is contingent upon meeting the qualification and requirements of the role, and for the successful completion of the interview selection and process.  It is at the Company's discretion to determine what pay is provided to a candidate within the range associated with the role.

Company Information

Company Name: Allied Benefit Systems, LLC

Company Website: https://www.alliedbenefitsystems.com

Company Address: 200 W Adams St, Suite 500 Chicago, IL 60606 United States

Allied Benefit Systems, LLC is a U.S.-based company that provides employer-focused benefits administration technology and outsourced benefits management services. The organization positions itself as a service and technology provider that helps employers, third‑party administrators, brokers and benefit consultants manage the lifecycle of employee benefits programs. Allied Benefit Systems combines web-based enrollment and benefits administration platforms with operational services such as billing, eligibility management, regulatory compliance support, and customer service to deliver an end-to-end solution for benefits administration. Core business activities: Allied Benefit Systems’ principal activities center on benefits administration technology and the operational processing that supports employer-sponsored health and welfare plans. The company develops and operates online portals and software tools for open enrollment, qualifying event processing, and ongoing employee self-service. It provides automated eligibility and census management to ensure accurate carrier billing and premium reconciliation, and it typically offers connectivity and integration between an employer’s human resources information system (HRIS), payroll systems, insurance carriers, and third-party vendors. Allied Benefit Systems also commonly delivers administration for pre-tax accounts (such as flexible spending accounts (FSAs) and commuter benefits), health reimbursement arrangements (HRAs), and health savings accounts (HSAs) either directly or through partner networks. Main products and services: The company’s technology suite is built around a secure, web-based benefits administration platform that supports online enrollment workflows, decision-support tools for employees, dependent verification processes, and document management. Key platform capabilities include configurable benefit elections and plan setup, dependent and beneficiary management, automated qualifying life event (QLE) handling, COBRA administration and notices, standardized reporting and analytics, and rule-based eligibility processing. The platform is typically offered with multi-channel access (employee self-service web portal, mobile-friendly interface, and administrator dashboards) and role-based access controls for HR teams and brokers. Allied Benefit Systems complements its technology with a range of professional services. These often include implementation and data migration when onboarding new employer clients, carrier and payroll file management, reconciliation and remittance services, and call-center or concierge-level participant support to handle employee inquiries and enrollment assistance. The company typically supports compliance-related deliverables such as ACA tracking and reporting, distribution of required notices, and assistance with COBRA and HIPAA communications. Many clients rely on Allied Benefit Systems for monthly billing and premium collection processing, as well as for consolidated invoicing when employers maintain multiple carriers or funding arrangements. Technology integrations and interoperability: A distinguishing characteristic of Allied Benefit Systems’ offering is integration capability. The company’s platform is designed to exchange data with HRIS systems, payroll vendors, insurance carriers, pharmacy benefit managers (PBMs), and third-party vendors for retirement and ancillary benefits. Integrations are commonly delivered via batch file feeds, secure FTP, or modern API endpoints, allowing eligibility and election changes entered during enrollment to flow automatically to payroll and carrier systems. The vendor typically provides standardized carrier file formats as well as custom mapping services to align with an employer’s existing ecosystem of benefits vendors. Security, compliance and data handling: Allied Benefit Systems places emphasis on secure data handling practices and compliance support for regulated benefits programs. Its technology environment commonly includes features such as encrypted data transmission, secure authentication, audit trails for enrollment changes, and role-based access control to protect sensitive personal health information and personally identifiable information. Compliance-focused services include assistance with ACA reporting, COBRA notices and administration, HIPAA privacy and security coordination, and support for dependent eligibility verification processes. Target customers and market positioning: Allied Benefit Systems primarily serves mid-sized and large employers, broker and consultant groups, and third-party administrators seeking an outsourced or hybrid model for benefits management. The platform and services are positioned to reduce administrative burdens on internal HR teams, decrease errors in eligibility and billing, and improve participant experience through streamlined enrollment and benefit education tools. Brokers and benefits consultants often use Allied Benefit Systems’ portal to manage multiple client accounts and to provide white-label enrollment experiences for employer clients. Operational model and service delivery: Allied Benefit Systems combines software-as-a-service (SaaS) delivery for its enrollment and administration platform with managed services delivered by trained benefits specialists. Implementation services typically include configuration of benefit plans, employee and dependent data onboarding, benefit election testing, and carrier file mapping. Ongoing managed services may include monthly billing reconciliation, carrier premium remittance, participant support via a service center, and periodic reporting and analytics to help clients monitor enrollment trends, cost drivers, and compliance metrics. Customer support and service levels: The company generally offers client support channels that include a dedicated implementation team, ongoing account management, and participant call-center support. Service-level agreements (SLAs) are usually negotiated based on the scope of services, with turnaround times defined for standard administrative tasks, electronic file delivery, and issue resolution. Training and client enablement services are often provided to ensure HR administrators and brokers can effectively use the administration portal and reporting tools. Business differentiation: Allied Benefit Systems distinguishes itself through the combination of its benefits administration platform and a comprehensive managed services model. By integrating enrollment technology, carrier connectivity, and hands-on administrative support, the company seeks to simplify complex benefits ecosystems for employers and brokers. Its ability to provide custom configurations, carrier file management, and consolidated billing are typical differentiators for organizations needing a single point of coordination across multiple benefits vendors. Note: The description above summarizes the company’s common offerings and positioning as a benefits administration and HR technology provider. Specific product names, certifications, pricing, geographic coverage, and partner relationships vary by contract and over time; for the most current and detailed information, consult the company’s official website or contact their sales representatives directly.
Visit company vacancy