Scaling Expert Oversight of AI: HackerOne's Human-in-the-Loop Methodology

Human-in-the-loop (HiTL) validation is a common practice in the responsible deployment of AI. It ensures the output of a model is accurate, safe, and that the model continually improves. When the methodology is applied conventionally, the AI models and agents become more accurate and useful over time through feedback loops.

This article focuses on the HiTL validation techniques that underpin the HackerOne Code product and how we built a scalable support system, operating 24/7, capable of completing thousands of manual validations per day.

HackerOne Code’s HiTL Methodology

Code, HackerOne’s AI code security product, deploys a task-performance via human-AI collaboration HiTL methodology — where AI agents’ thought processes from code analysis are distilled into Explainable AI (XAI) to assist real human experts in validating it. 

You can think of it like having a teammate who encountered a tricky problem. Instead of saying “I don’t know” or making a best effort guess, they ask you for help. When they do, they catch you up by explaining what they figured out so far, their reasoning based on what they understand, and parts they’re unsure about.

We use this HiTL methodology because of the nature of HackerOne Code’s multi–agent orchestration for code scanning. HackerOne Code doesn’t just read and interpret code to detect security flaws, it scrutinizes the detections it does find to make sure they matter and are actionable in ways that pattern-match cognitive reasoning of a human engineer working in a continuous threat exposure management (CTEM) workflow. This is a complex task for an AI system to take on and the stakes are high, so a strong oversight layer is critical. Expert HiTL is qualified for the job because it complements agents with human knowledge, can impose guardrails based on judgment, and enable agents to reach an uncertain conclusion with an escalation channel.  In other words,  HiTL oversight is used to ensure agents interpret and act the way a human expert would. And the end-user experience feels like working with a teammate, not a bot.

A key variable in Code’s self-scrutinizing analysis is its confidence in its own conclusion. Issues that don’t meet or exceed a confidence threshold (which is high) are escalated for manual intervention. In this capacity, reasoning and context analysis from code scanning agents are distilled into decision-making aids for human experts. In other words, agents provide evidence for a human to check it.

Feedback Loops

Where HackerOne Code’s methodology is slightly different from conventional use of HiTL validation is that security teams and developers are the primary beneficiary of every HiTL touchpoint, not the AI system. This means that when our experts conduct manual validation their focus is entirely on helping the developer who wrote the code. The AI system becomes smarter and more accurate by “shadowing” human experts working together.

HiTL validation creates feedback loops in two categories:

• System-specific accuracy with memories: A loop for creating long-term memories for information agents can only reference when working within the same system context. So it can be more accurate and useful when working with the same system.
• Cross-system, generalized: Learnings based on patterns of behaviors and outcomes that improve accuracy and utility without context of the system. So it becomes smarter on general concepts.

Memories

Memories make our AI agents smarter and more helpful working with specific teams on the things they work on, the knowledge they can only know and take into account working in that specific context. This feedback loop is what makes HackerOne Code learn like a teammate.

By listening and learning, HackerOne Code becomes smarter about things like architecture, institutional knowledge, policy, and priorities. Dialog transpiring from HiTL validation (e.g., confirming assumptions, unintuitive edgecases) is a high-signal source of information valuable for it to know.

Memories are salient artifacts, so human actors (HackerOne engineers, security teams and developer users) can create them and interact with them directly. HackerOne Code allows security teams and developers to invoke HackerOne Code and point to specific information it should know.

Cross-system, Generalized

HiTL validation also creates a valuable feedback loop to guide our research and development priorities. Where memories inform our AI agents on what they need to know analyzing code for a specific team, these feedback loops provide general knowledge for fine-tuning our AI systems. Learnings come from behaviors of human experts exercising judgment. These are the feedback loops that help HackerOne Code think and work like a developer.

Examples:

• Classes of vulnerabilities that are challenging to validate or take more time highlight detection agents that need fine-tuned guidance.
• Certain types of security issues that are often validated, have a predicted “High” severity, but a lower than average fix rate may need to be fine-tuned so recommendations are more actionable.
• If HackerOne engineers spend significantly more time validating issues in a specific programming language, our context helper system may need to be further conditioned for how to navigate its architecture paradigms.

The Importance of Speed

On top of code security analysis being a high-stakes, complex task, it needs to be done quickly. This is critical not only for protecting velocity, but capitalizing on the window of time context of code changes is best known.

To achieve this, we purpose built our HiTL validation system for speed and obsessively optimized to reduce cognitive load. This allows engineers validating our agents to catch up on a lot of context and reach decisions quickly. In a majority of cases, security issues that require manual review validation are completed within 90 minutes of a pull or merge request being opened.

The Importance of Expertise

Security risk validation is complex and nuanced. This kind of AI oversight can’t be done by just anyone. The role of HITL validation needs an expert.

At HackerOne, our secure code AI technology is overseen by a network of over 600 expert engineers. Adhering to a strict selection and eligibility process, members range from open source maintainers to technical co-founders — with expertise coverage for all major frameworks, libraries and programming languages. 

Building this network to what it is today wasn’t easy, but we believe human oversight is a non-negotiable core tenet of responsible AI.

Taking a page from similar strategies we learned creating a community of over 2 million security researchers, here’s how we’ve built a network of HackerOne engineers for scaling HiTL validation.

Eligibility & Selection

Validation of security code flaws in code is a highly skilled task. So our eligibility standards are high to match the stakes at hand. Less than 2% of applicants advance through the initial screening process.

Professional Experience

To understand, validate, and help remediate security flaws in code, you need a career in application security and software development looking at lots of it. With rare exception, HackerOne engineers conducting validation for Code have at least 5 or more years professional experience in senior-level engineering roles. This can vary based on types of programs and business contexts which they specialize in.

Some of the key criteria we look for:

• Experience as a decision-maker for system design.
• Impact on notably complex or technical projects.
• Adaptability; an awareness of reasonable solutioning when the optimal implementation is impractical.

Meeting the Core Team

All members of our HiTL network spend time meeting the team - on video or in person. We use these opportunities to understand the engineer’s professional experience, get to know their notable strengths, and record domains to knowledge used for kill categorization.

It’s also critical for understanding their style of communication and team cultures they work best with. Experts must be able to effectively collaborate with developers through written communication, so productive dialog is a critical factor in making sure security risks are known so they can be fixed. In some cases, complex engineering paradigms need to be explained through pull or merge request messages.

Location

Over 95% of HackerOne’s engineers who complete HiTL validation are based in the United States, with a few members in the United Kingdom, Canada, Australia and New Zealand. This is because we need to ensure work is done in countries with strong intellectual protection and rule of law to exercise proper diligence in enterprise environments.

Trust

All members complete a criminal background check, on-disclosure, confidentiality, intellectual property as well as proprietary information and inventions assignment before beginning work as a HackerOne contractor.

Skill & Expertise Matching

Technology-level expertise of each member is maintained using over 140 categories. These identify programming languages, frameworks, libraries, and platforms. They include:

General-purpose languages: 

Python, Java, C, C++, Ruby, PHP, Swift, Kotlin, etc.

Front-end frameworks and libraries: 

React, Angular, AngularJS, Backbone, etc.

Databases: 

MySQL, SQL, Redis, DynamoDB, etc.

Cloud and DevOps:

CDK, CloudFormation, Jenkins, GitLabCI, etc.

Frameworks and Libraries:

SpringMVC, Material-UI, Flutter, ASP.NET, etc.

Testing, API & Protocols:

Cucumber, Gherkin, Swagger, IAM, etc.

With a skillset of each member, a relative scale of proficiency is applied (e.g., an application security engineer who is highly experienced working with applications written in React, Laravel and PHP but relatively new to working with applications built with Angular and Go).

Advanced Matching

Knowing which experts in our network are the best fit for specific codebases and teams doesn’t stop at what kind of code is used. For example, consider an engineer proficient in Rust - a programming language that emphasizes performance, safety, and concurrency. Rust is a popular choice for everything from low-level embedded systems to GraphQL APIs to Web3 protocols and infrastructure. As part of an organization’s attack surface, the security implications for these can vary a lot. So matching human experts based on type of code alone isn’t enough.

That’s why, along with our Context Engine which aids in understanding business logic and intent, we match experts based on a comprehensive profile of experience and believability.

Here are a few examples of additional factors beyond code type taken into account for expert eligibility and selection:

• Industry verticals an expert has notable experience in.
• Exposure to software use cases spanning commercial, academic and government.
• If the code is open source with an active contributor community.
• Whether architectural decisions are still being made for the codebase or have been established for years.

By matching experts based on a full knowledge of their professional experience and contextual awareness of the codebase, we can find the very best people to perform manual code review validation and give developers the best experience. 

Behind Every AI Agent, a Security Expert You Can Trust

At HackerOne, we believe while AI is a powerful tool to achieve consistency and scale, it’s no replacement for human expertise. That's why our methodology combines both—AI with human expertise to catch security vulnerabilities in AI-generated or human-written code and help development teams fix them. We provide fast, actionable feedback that’s non-blocking and actively supports engineers throughout remediation. To do this, we use a multi-agent orchestration for analyzing code with a scalable HiTL methodology. Code security AI agents are overseen by over 600 human experts. 

HackerOne engineers who oversee the AI layer are carefully vetted and matched with codebases for manual code review and validation based on a comprehensive profile of experience and believability.