crawler.response becomes null when connecting to specific chrome instance

Created by: carlkentor

What is the current behavior? Okay, I dont know if this should be classified as a bug or not. But since I cant seem to find any trace of this in the docs, here it is.

When connecting to a specific chrome instance through the: await HCCrawler.connect({ browserWSEndpoint: wsChromeEndpointurl, ...more options goes here}) method, the call to await crawl() results in a null response. The issue lies within:

\node_modules\headless-chrome-crawler\lib\crawler.js

async crawl(){
await this._prepare();
const response = await this._request(); //1. Results in Null
console.log(response); //2. ======> NULL INDEED
await this._waitFor();
const [
  result,
  screenshot,
  cookies,
  links,
] = await Promise.all([
  this._scrape(),
  this._screenshot(),
  this._getCookies(),
  this._collectLinks(response.url),  //3. ====> Exception gets thrown here 
]);
return {
  options: this._options,
  depth: this._depth,
  previousUrl: this._previousUrl,
  response: this._reduceResponse(response),
  redirectChain: this._getRedirectChain(response),
  result,
  screenshot,
  cookies,
  links,
}; }

Steps to reproduce

Close all chrome instances.
Open CMD and run the following to start Chrome with remote debugging enabled:

start chrome.exe --remote-debugging-port=9222

Navigate to http://localhost:9222/json/version and copy the value of "webSocketDebuggerUrl"
Set the browserWSEndpoint to the value you just copied.

What is the expected behavior? When running through a clean instance with the

await HCCrawler.launch({
    ...options
})

command, everything works fineand the result comes back as expected. What am I missing here?

Please tell us about your environment:

Version: 1.8.0
Platform / OS version: Windows 10 / 1803
Node.js version: v10.16.3