Dr. Heather Leffew

Founding leader of Spokeo's Data Science organization, partnering with Product, Engineering, Legal, Compliance, and Executive Leadership to make production AI and ML systems measurable, auditable, and useful across identity, search, and conversational products.

• Built and deployed evaluation frameworks for entity resolution, social identity, ranking, and conversational AI systems, including weak supervision, LLM-as-a-Judge, multi-model adjudication, and human-in-the-loop evaluation, grounded in construct validity and measurement theory.
• Developed replay-based evaluation methodologies, outcome-based measurement frameworks, and telemetry infrastructure to identify model failure modes, ranking opportunities, compliance risks, user-friction patterns, recommendation opportunities, and product quality improvements.
• Designed custom agentic harnesses for AI-assisted workflows, embedding auditable review processes, evaluation evidence, and quality controls directly into engineering workflows.
• Implemented review standards focused on measurement integrity, construct validity, and decision quality across applied ML and AI systems.
• Led compliance evaluation initiatives across AI systems and monitoring workflows, supporting zero confirmed compliance failures across reviewed production samples and ISO/IEC 42001 readiness through AI governance, auditability, model oversight, and Responsible AI operating standards.
• Built automated source-evaluation frameworks measuring coverage, quality, signal contribution, and ROI, guiding multi-million-dollar data acquisition and renewal decisions.

Led applied ML research, evaluation strategy, and platform-integrity measurement across Trust & Safety, brand safety, election integrity, minor safety, and moderation systems. Partnered with Data Science, Engineering, Policy, and Operations teams to turn complex safety questions into measurable detection, evaluation, and governance workflows.

• Built ML measurement and detection systems across NLP, computer vision, behavioral analytics, and distributed data processing to improve safety evaluation at platform scale.
• Developed enforcement measurement frameworks that helped teams distinguish model precision from over-enforcement risk, supporting higher-confidence actioning and review workflows.
• Built election-integrity analytics that unified search, content, engagement, and moderation data into a shared evidence layer for cross-functional decision-making.
• Enhanced minor-safety analysis with behavioral network discovery, helping teams understand how risk emerges across accounts, surfaces, and time.
• Built distributed PySpark pipelines over nested moderation events to support user-level safety analysis, review prioritization, and forensic reporting with necessity, proportionality, accountability, and redaction guardrails.
• Applied survival analysis, anomaly detection, sentiment drift, topic modeling, and behavioral trajectory analysis to measure safety degradation and user-experience risk.
• Established external IRB methodology for research partnerships and mentored researchers on statistics, measurement, methodology, and research design.

Led accreditation, evaluation, and product operations for a public-safety training and certification organization. Managed the full accreditation lifecycle and partnered with agencies, government stakeholders, and internal teams to translate standards, operational requirements, and evidence into defensible evaluation programs.

• Built standards-aligned assessment and review frameworks for emergency responder training, certification, renewal, and site-visit processes.
• Scaled accreditation programs to support 1,000+ active agencies through structured evaluation workflows, documentation standards, and repeatable decision criteria.
• Leadership and P&L ownership across product development, market analysis, agency evaluation, and client delivery.
• Expanded B2B client acquisition and recurring revenue by authoring RFP responses that translated agency requirements into practical product and evaluation scope.
• Guided leadership decisions in regulated public-safety environments by presenting risk analyses, performance metrics, program-quality evidence, and accreditation findings to government officials and oversight bodies.

Directed evaluation, analytics, and digital product delivery for regulated public-safety psychology programs. Owned assessment products, people analytics platforms, wellness programs, research studies, and program-evaluation frameworks for law enforcement and emergency-response clients.

• Built evaluation frameworks for psychological assessment, public-safety interventions, wellness programs, and organizational risk measurement.
• Led three product teams totaling 18 members across assessment delivery, analytics, digital transformation, and client-facing program operations.
• Increased processing speed 5x by replacing legacy workflows with cloud-based assessment products, digitized operations, and standardized analytics processes.
• Secured $500K+ in grant capital by building data-driven roadmaps that demonstrated measurable DEI outcomes grounded in behavioral science.
• Forecasted mental health service needs using predictive analytics pipelines and evidence-based measurement.
• Expanded a multi-state client portfolio by aligning psychological services with regulatory expectations, agency procurement needs, and required RFP standards.
• Applied implicit behavioral measurement to pre-employment fitness evaluations, occupational risk assessment, and high-stakes public-safety decision support.

Led supervision and training for 12 master's-level clinicians and practicum students across clinical and public safety settings, while conducting research in predictive behavioral analytics and clinically grounded measurement.

• Diagnostic Psychological Evaluations: Cedar Springs Psychiatric Hospital.
• Forensic, Occupational, and Clinical Neuropsychological Assessment: Rocky Mountain Behavioral Health.
• Critical Incident Response, Fitness for Duty Evaluations, Psychotherapy: Brower Psychological Police and Public Safety Psychology.
• Assistant Disaster Coordinator: Comprehensive Clinical Services (Aurora Mental Health Center).
• Teaching Assistant: Theories of Psychotherapy.
• Research Assistant: Clinically Predictive Linguistic Analysis of Thematic Apperception Test Narratives.

Built analytics and audit products for private-sector hiring, workforce assessment, fairness review, and FCRA-compliant HR decision support. Partnered with B2B clients to identify systemic bias risks, strengthen assessment quality, and translate measurement findings into repeatable governance and remediation workflows.

• Designed analytics products to evaluate hiring practices, workforce selection systems, adverse impact, and diversity outcomes.
• Established psychometric and statistical standards for talent-assessment audits, including construct validity, fairness measurement, and defensible decision criteria.
• Implemented bias-mitigating policies at scale that achieved full FCRA compliance for enterprise HR partners.
• Generated recommendations that improved diversity outcomes through repeatable statistical modeling, audit workflows, and evidence-based governance practices.
• Built the data-science foundation for later Responsible AI work by connecting model outputs, regulated decision systems, fairness risk, and human assessment validity.