Scratch Tesla Stock Prices with Node.js and Puppeteer

0
picture
Code_Jedi Hacker Noon's profile picture

@codejediCode_Jedi

Python, Machine Learning, Web-Scraping, Web-Automation and more …

Hi, developer friends

In this tutorial, I will show you how to mine the latest Tesla stock prices using Node.js and Puppeteer.

Let’s get started!

First, you will need to install the puppeteer using

npm i puppeteer

. Now if you don’t

npm

,

package.json

, and

node_modules setup

, here is a great tutorial.

After installing puppeteer, create a new javascript file and require puppeteer on the first line:

const puppeteer = require('puppeteer');

Then create the async function in which we will write our main code:

const puppeteer = require('puppeteer');
async function start() {
}
start();

We are now ready to start scratching.

First, you need to launch a new browser instance, as well as define the url your web-scraper will visit:

const puppeteer = require('puppeteer');
async function start() {
  const url = 'https://finance.yahoo.com/quote/TSLA?p=TSLA&.tsrc=fin-srch';
  const browser = await puppeteer.launch({
    headless: false
  });
}

Then you need to call the

newPage()

function to open a new page in the browser, and go to the url we defined using the

goto()

function:

const puppeteer = require('puppeteer');
async function start() {
  const url = 'https://finance.yahoo.com/quote/TSLA?p=TSLA&.tsrc=fin-srch';
  const browser = await puppeteer.launch({
    headless: false
  });
  const page = await browser.newPage();
  await page.goto(url);
}

For this next step, you will need to go to https://finance.yahoo.com/quote/TSLA?p=TSLA&.tsrc=fin-srch, right click on the current share price, then click on inspect:

picture

A pop-up will appear to the right of your window, you will need to find the price of action item:

picture

Then you will need to right click on the stock price item and click on ‘copy full xpath’.
This will give us a way to access the share price element:

picture

Once we have the Xpath of the price of action item, we can add these 3 lines of code into our function:

var element = await page.waitForXPath("put the stock price Xpath here")
  var price = await page.evaluate(element => element.textContent, element);
  console.log(price);

The

page.waitForXPath()

function will locate the price element of the stock.
Then the

page.evaluate

The function will get the textual content of the stock’s price element which would then be printed by the

console.log()

function.

At this point, your code should look like this:

const puppeteer = require('puppeteer');
async function start() {
  const url = 'https://finance.yahoo.com/quote/TSLA?p=TSLA&.tsrc=fin-srch';
  const browser = await puppeteer.launch({
    headless: false
  });
  const page = await browser.newPage();
  await page.goto(url);
  var element = await page.waitForXPath("/html/body/div[1]/div/div/div[1]/div/div[2]/div/div/div[5]/div/div/div/div[3]/div[1]/div[1]/span[1]")
  var price = await page.evaluate(element => element.textContent, element);
  console.log(price);
}
start()

If you were to run your current code, you would find that by going to the url you set earlier, a pop-up window will appear:

picture

To work around this problem, plug these 2 lines of code into your function before defining the “element” variable:

var accept = ("#consent-page > div > div > div > form > div.wizard-body > div.actions.couple > button");
await page.click(accept)

This will locate the “Accept All” button and click on it to make the pop-up disappear.

Now you will have a working function that goes to your defined URL, scratches the latest Tesla share price, and prints it to your terminal.

To go further, you can put these lines of code in a for loop:

for(var k = 1; k < 2000; k++){
      var element = await page.waitForXPath("/html/body/div[1]/div/div/div[1]/div/div[2]/div/div/div[5]/div/div/div/div[3]/div[1]/div[1]/span[1]")
      var price = await page.evaluate(element => element.textContent, element);
      console.log(price);
      await page.waitForTimeout(1000);
    }

The

page.waitForTimeout(1000)

The function will wait 1000 milliseconds (1 second) before repeating the for loop.

And finally,

add a browser.close()

function after the for loop to close the browser and finish executing your code:

const puppeteer = require('puppeteer');
async function start() {
    const url = 'https://finance.yahoo.com/quote/TSLA?p=TSLA&.tsrc=fin-srch';
    const browser = await puppeteer.launch({
      headless: false
    });  
    const page = await browser.newPage();
    await page.goto(url);
    var accept = ("#consent-page > div > div > div > form > div.wizard-body > div.actions.couple > button");
    await page.click(accept);
    for(var k = 1; k < 2000; k++){
      var element = await page.waitForXPath("/html/body/div[1]/div/div/div[1]/div/div[2]/div/div/div[5]/div/div/div/div[3]/div[1]/div[1]/span[1]");
      var price = await page.evaluate(element => element.textContent, element);
      console.log(price);
      await page.waitForTimeout(1000);
    }
    browser.close();
}
start();

Get the hottest lineup: Tweets, Reddit posts, Github Rests, and memes of the week straight to your inbox every Friday via my newsletter!

Bye

Als published here.

Code_Jedi Hacker Noon's profile picture

Key words

Join Hacker Midi

Create your free account to unlock your personalized reading experience.



Source link

Share.

Leave A Reply