.each function returns just one element

node , javascript , scraper United States
  • 3 months ago

    The function imgArray returns just one image and not an array. That is, the third column of the spreadsheet which is img lists just one image. Here is my code:

    const rp = require('request-promise');
    const otcsv = require('objects-to-csv');
    const cheerio = require('cheerio');
    
    const baseURL = 'https://www.example.com';
    
    const getCategories = async () => {
    const html = await rp(baseURL);
    
    const imgArray = () => {
    cheerio('td.productListing-data > a > img', html).each((i, image) => {
     img = cheerio(image).attr('src');
    })};
    
    imgArray();
    
      const businessMap = cheerio('.category', html).map(async (i, e) => {
        const link = e.attribs.href;
        const innerHtml = await rp(link);
        const cat = e.children[0].data;
    
        return {
          link,
          cat,
          img,
        }
      }).get();
      return Promise.all(businessMap);
    
    };
    
    getCategories()
    
    
      .then(result => {
        const transformed = new otcsv(result);
        return transformed.toDisk('./spreadsheets/output.csv');
      })
      .then(() => console.log('SUCCESSFULLY COMPLETED THE WEB SCRAPING SAMPLE'));
    
  • 3 months ago

    I changed the code as follows and it still just gives me one item:

    l et img; const images = cheerio('td.productListing-data > a > img', html).map(async (i, image) => { img = cheerio(image).attr('src'); }).get();

  • 3 months ago
    let img;
    const images = cheerio('td.productListing-data > a > img', html).map(async (i, image) => {
    img = cheerio(image).attr('src');
    }).get();
    
  • 3 months ago

    My code was incorrectly formatted and I didn't have the option to edit it so I just re-entered it above.

  • 3 months ago

    I used:

    const images = cheerio('td.productListing-data > a > img', html)
      .map((i, image) => cheerio(image).attr('src'))
      .get()
    

    that had the result that every cell of the spreadsheet contained an array.

  • 3 months ago

    I tried iterating through the array but couldn't make it work with the return statement.

Post a reply

Enter your message below

Sign in or Join us (it's free).

Contribute

Why not write for us? Or you could submit an event or a user group in your area. Alternatively just tell us what you think!

Our tools

We've got automatic conversion tools to convert C# to VB.NET, VB.NET to C#. Also you can compress javascript and compress css and generate sql connection strings.

“Every language has an optimization operator. In C++ that operator is ‘//’”