SBSD Challenge Flow

Understanding SBSD Protection

State Based Scraping Detection (SBSD) is an advanced bot protection mechanism used by Akamai to protect websites from scrapers. Websites protected by SBSD present a challenge that must be solved before allowing access to the protected content.

Challenge Flow Overview

The SBSD challenge follows a specific sequence of requests that must be executed in order:

Initial Page Request: Client makes a request to the protected website
Challenge Page Response: Server returns a challenge page with a script reference
Script Request: Client fetches the referenced script
Payload Construction & Submission: Client generates and submits the required payload
Access Grant: Upon successful verification, access to the website is granted

Implementation Guide

Step 1: Initial Page Request

Make a standard GET request to the target website. When the site is protected by SBSD, instead of receiving the actual content, you'll receive a challenge page.

<html>
   <body>
      <script src="/6mGXhhKgo3Cn/HH/EB0WcpIr3K/X0iuwmY3aY/UkZyWg/Dy/J1CmB4HUQ?v=99b02ce6-f91f-0f49-40ae-6f8493e30211&t=183446611"></script>
      <script>
         (function() {
             var chlgeId = '';
             var scripts = document.getElementsByTagName('script');
             for (var i = 0; i < scripts.length; i++) {
                 if (scripts[i].src && scripts[i].src.match(/t=([^&#]*)/)) {
                     chlgeId = scripts[i].src.match(/t=([^&#]*)/)[1];
                 }
             }
             var proxied = window.XMLHttpRequest.prototype.send;
             window.XMLHttpRequest.prototype.send = function() {
                 var pointer = this
                 var intervalId = window.setInterval(function() {
                     if (pointer.readyState === 4 && pointer.responseURL && pointer.responseURL.indexOf('t=' + chlgeId) > -1) {
                         location.reload(true);
                         clearInterval(intervalId);
                     }
                 }, 1);
                 return proxied.apply(this, [].slice.call(arguments));
             };
         })();
      </script>
   </body>
</html>

Step 2: Identify Challenge Signature

The response will contain an HTML page with a script tag that has essential parameters for solving the challenge. You need to extract:

Path: The script URL path
v parameter: A UUID value
t parameter: A challenge token

Example script tag:

<script src="/6mGXhhKgo3Cn/HH/EB0WcpIr3K/X0iuwmY3aY/UkZyWg/Dy/J1CmB4HUQ?v=99b02ce6-f91f-0f49-40ae-6f8493e30214&t=183446612"></script>

From this example:

Path: /6mGXhhKgo3Cn/HH/EB0WcpIr3K/X0iuwmY3aY/UkZyWg/Dy/J1CmB4HUQ
v parameter: 99b02ce6-f91f-0f49-40ae-6f8493e30214
t parameter: 183446612

Implement a regular expression to extract these values:

const regex = /([a-z\d/\-_\.]+)\?v=(.*?)(?:&.*?t=(.*?))?["']/i;
const matches = html.match(regex);
const path = matches[1];
const v = matches[2];
const t = matches[3] || "";

Step 3: Fetch Challenge Script

Request the script using the extracted components:

GET /[path]?v=[v_parameter]&t=[t_parameter] HTTP/2
Headers...

You'll need to save this script content for use in the next step.

Step 4: Generate and Submit Payload

Using our API service, generate the SBSD payload by providing:

The extracted UUID (v parameter)
The page URL
The script content
Your User-Agent
Any existing sbsd_o cookie value, or the bm_so cookie value if sbsd_o is not present.

POST /sbsd HTTP/1.1
Content-Type: application/json

{
  "userAgent": "Mozilla/5.0...",
  "uuid": "99b02ce6-f91f-0f49-40ae-6f8493e30214",
  "pageUrl": "https://example.com/",
  "o": "existing_sbsd_o_cookie_value_if_any",
  "script": "script_content_from_step_3",
  "ip": "your ipv4 or ipv6 address",
  "acceptLanguage": "en-US,en;q=0.9"
}

Our API will return the properly formatted payload string.

Step 5: Submit the Solution

POST the generated payload to the challenge endpoint:

POST /[path]?t=[t_parameter] HTTP/2
Headers...

{
  "body": "YOUR_GENERATED_PAYLOAD"
}

Step 6: Access Protected Content

If the payload is correct, you can now make requests to the protected website and receive the actual content instead of the challenge page.

GET / HTTP/2
Headers...

Implementation Example

Here's a pseudocode example showing the complete flow:

// Step 1: Initial request
const initialResponse = fetch("https://example.com/");
const html = await initialResponse.text();

// Step 2: Extract challenge parameters
const regex = /([a-z\d/\-_\.]+)\?v=(.*?)(?:&.*?t=(.*?))?["']/i;
const matches = html.match(regex);
const path = matches[1];
const v = matches[2];
const t = matches[3] || "";

// Step 3: Fetch script
const scriptUrl = `https://example.com${path}?v=${v}&t=${t}`;
const scriptResponse = await fetch(scriptUrl);
const scriptContent = await scriptResponse.text();

// Step 4: Generate payload using our API
const payload = await generatePayload({
  userAgent: "YOUR_USER_AGENT",
  uuid: v,
  pageUrl: "https://example.com/",
  oCookie: getCookie("sbsd_o"),
  script: scriptContent,
  ip: yourIp,
  acceptLanguage: yourAcceptLanguage
});

// Step 5: Submit payload
const submitUrl = `https://example.com${path}?t=${t}`;
await fetch(submitUrl, {
  method: "POST",
  headers: {
    "Content-Type": "application/json"
  },
  body: JSON.stringify({ body: payload })
});

// Step 6: Access protected content
const protectedContent = await fetch("https://example.com/");

Important Notes

Keep Headers Consistent: Align your headers with Chrome using a TLS client.
Header Order Matters: Akamai verifies header ordering on all requests
Cookie Management: Properly store and reuse any cookies set by the server

Integration with Our API

Our API simplifies the most complex part of this process - generating the correct payload. By providing the necessary parameters to our service, you receive a properly formatted payload ready for submission.

For detailed integration instructions and API endpoints, refer to our API Reference Documentation.

PreviousSBSD Introduction NextHandling 429 Status Codes with SBSD Challenges

Last updated 7 months ago

hashtagUnderstanding SBSD Protection

hashtagChallenge Flow Overview

hashtagImplementation Guide

hashtagStep 1: Initial Page Request

hashtagStep 2: Identify Challenge Signature

hashtagStep 3: Fetch Challenge Script

hashtagStep 4: Generate and Submit Payload

hashtagStep 5: Submit the Solution

hashtagStep 6: Access Protected Content

hashtagImplementation Example

hashtagImportant Notes

hashtagIntegration with Our API