Scrape Text From Website

By
 import os
import datetime
import csv
from flask import Flask, request, render_template, send_file
from bs4 import BeautifulSoup
import requests
import tempfile

app = Flask(__name__)

@app.route("/", methods=["GET", "POST"])
def root_route():
    if request.method == "POST":
        url = request.form["url"]
        try:
            response = requests.get(url)
            soup = BeautifulSoup(response.text, 'html.parser')
            texts = soup.stripped_strings
            scraped_text = " ".join(texts)
            date_scraped = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
            data = [{"Source URL": url, "Scraped Text": scraped_text, "Date Scraped": date_scraped}]
            # Generate CSV content
            csv_content = "Source URL,Scraped Text,Date Scraped\\n"
            csv_content += f"\"{url}\",\"{scraped_text}\",\"{date_scraped}\"\\n"

About this template

A web app that allows users to input a URL and scrape the text from any webpage, displaying it in a formatted table along with the source URL and date scraped. Users can also download the table as a CSV file.

Introduction to the Web Scraper Pro Template

Welcome to the Web Scraper Pro template guide. This template allows you to create a web application that can scrape text from any webpage. Users can input a URL, and the app will display the scraped text in a formatted table along with the source URL and the date scraped. Additionally, users have the option to download this data as a CSV file. This guide will walk you through the steps to set up and use this template on the Lazy platform.

Getting Started with the Template

To begin using the Web Scraper Pro template, simply click on "Start with this Template" on the Lazy platform. This will pre-populate the code in the Lazy Builder interface, so you won't need to copy or paste any code manually.

Test: Deploying the App

Once you have started with the template, the next step is to deploy the app. Press the "Test" button on the Lazy platform. This will initiate the deployment process and launch the Lazy CLI. The app will be deployed on the Lazy platform, and you won't need to worry about installing libraries or setting up your environment.

Entering Input: Providing the URL

After pressing the "Test" button, if the app requires user input, the Lazy App's CLI interface will appear. You will be prompted to provide the URL of the webpage you want to scrape. Enter the URL when prompted to proceed with the scraping process.

Using the App: Interacting with the Web Interface

Once the app is running, you will be able to interact with the web interface. The main page will present you with a form where you can enter the URL of the webpage you wish to scrape. After submitting the URL, the app will scrape the text and display it in a table format on a new page. You will also see an option to download the data as a CSV file.

Integrating the App: Using the Scraped Data

If you wish to integrate the scraped data into another tool or service, you can use the CSV file that the app generates. Download the CSV file and import it into your desired tool for further analysis or processing.

Remember, this template is ideal for full-stack web applications that require interactive elements on the front end while leveraging Python's robust backend capabilities. It is not suitable for backend-only applications.

By following these steps, you should be able to successfully set up and use the Web Scraper Pro template on the Lazy platform. Enjoy building your web scraping application!

Category
Last published
April 7, 2024

More templates like this

Customisable Server Status Dashboard

This template allows you to create a dashboard to report about incidents regarding your server to your users.

Streamlit
Python

A bot that answers questions about data

Ask questions about your database via a chat bot. This chatbot connects to a SQLITE database, generates queries for it based on the schema and then runs the queries printing the response all in a nicely styled chat UI. This template is pre-loaded with sample data (car parts) to play around with - here are the sample data columns that you can query: part_number, part_name, price, units_left_in_stock, manufacturer .

Python
Flask

WhatsApp Bot Builder

The WhatsApp Bot Builder app allows users to create a WhatsApp bot that responds to every message with "Hello World".

WhatsApp
Twilio
Python
Home
/
Scrape Text From Website