Web Scraper Pro

By
 import os
import datetime
import csv
from flask import Flask, request, render_template, send_file
from bs4 import BeautifulSoup
import requests
import tempfile

app = Flask(__name__)

@app.route("/", methods=["GET", "POST"])
def root_route():
    if request.method == "POST":
        url = request.form["url"]
        try:
            response = requests.get(url)
            soup = BeautifulSoup(response.text, 'html.parser')
            texts = soup.stripped_strings
            scraped_text = " ".join(texts)
            date_scraped = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
            data = [{"Source URL": url, "Scraped Text": scraped_text, "Date Scraped": date_scraped}]
            # Generate CSV content
            csv_content = "Source URL,Scraped Text,Date Scraped\\n"
            csv_content += f"\"{url}\",\"{scraped_text}\",\"{date_scraped}\"\\n"

About this template

Web Scraper Pro: A web app that allows users to input a URL and scrape the text from any webpage, displaying it in a formatted table along with the source URL and date scraped. Users can also download the table as a CSV file.

Introduction to the Web Scraper Pro Template

Welcome to the Web Scraper Pro template! This template is designed to help you create a web application that can scrape text from any webpage. The app allows users to input a URL, scrape the text from the page, and display the results in a formatted table. Additionally, users can download the data as a CSV file. This step-by-step guide will walk you through the process of using this template on the Lazy platform.

Getting Started with the Template

To begin building your web scraping application, click on "Start with this Template" on the Lazy platform. This will pre-populate the code in the Lazy Builder interface, so you won't need to copy or paste any code manually.

Test: Deploying the App

Once you have the template loaded, press the "Test" button to start the deployment of your app. The Lazy CLI will handle the deployment process, and you won't need to worry about installing libraries or setting up your environment.

Entering Input

After pressing the "Test" button, if the app requires any user input, the Lazy App's CLI interface will prompt you to provide it. For this template, you will be asked to enter the URL of the webpage you want to scrape.

Using the App

Once the app is deployed, you will be provided with a dedicated server link to interact with your new web scraping application. The app features a simple interface where you can enter the URL of the webpage you wish to scrape. After submitting the URL, the app will display the scraped text in a table format on the webpage. You will also have the option to download this data as a CSV file.

Integrating the App

If you wish to integrate the Web Scraper Pro app into another service or frontend, you may need to use the server link provided by Lazy. For example, you could embed the link in an iframe within another webpage or use it as part of a larger system that requires scraped data.

Remember, this template is ideal for creating interactive web applications that require both frontend and backend capabilities. It's not suitable for backend-only applications.

If you encounter any issues or need further assistance, the Lazy customer support team is here to help you make the most out of this template.

Happy building with the Web Scraper Pro template on Lazy!

Category
Last published
April 7, 2024

More templates like this

Customisable Server Status Dashboard

This template allows you to create a dashboard to report about incidents regarding your server to your users.

Streamlit
Python

A bot that answers questions about data

Ask questions about your database via a chat bot. This chatbot connects to a SQLITE database, generates queries for it based on the schema and then runs the queries printing the response all in a nicely styled chat UI. This template is pre-loaded with sample data (car parts) to play around with - here are the sample data columns that you can query: part_number, part_name, price, units_left_in_stock, manufacturer .

Python
Flask

WhatsApp Bot Builder

The WhatsApp Bot Builder app allows users to create a WhatsApp bot that responds to every message with "Hello World".

WhatsApp
Twilio
Python
Home
/
Web Scraper Pro