 import json
from abilities import llm_prompt
from fastapi import Request
import logging

import uvicorn
from fastapi import FastAPI
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
from fastapi.staticfiles import StaticFiles
from selenium_utils import SeleniumUtility

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

app = FastAPI()
templates = Jinja2Templates(directory="templates")
app.mount("/static", StaticFiles(directory="static"), name="static")

@app.get("/", response_class=HTMLResponse)
async def form_page(request: Request):
    return templates.TemplateResponse("form.html", {"request": request})

@app.post("/process-page-info", response_class=HTMLResponse)

About this template

Use this skeleton to use AI to crawl a website

Introduction to the AI Scraper Selenium App Template

Welcome to the AI Scraper Selenium App template! This template is designed to help you build a web scraping application that leverages the power of AI to extract and process information from web pages. With this template, you can submit a URL and a question, and the app will return a summary of the page related to your query. This is perfect for non-technical builders who want to create software applications without worrying about the complexities of deployment and environment setup.

Getting Started

To begin using this template, simply click on "Start with this Template" on the Lazy platform. This will pre-populate the code in the Lazy Builder interface, so you won't need to copy or paste any code manually.

Test: Deploying the App

Once you have started with the template, press the "Test" button to deploy your app. The Lazy CLI will handle the deployment process, and you will not need to install any libraries or set up your environment. If the code requires any user input, you will be prompted to provide it through the Lazy CLI after pressing the "Test" button.

Entering Input

After deployment, if the app requires any user input, the Lazy CLI will prompt you to enter the necessary information. For this template, you will need to input the URL of the web page you want to scrape and the question you want to ask about the page content.

Using the App

Once the app is running, you will be provided with a dedicated server link to interact with the app's API. If you're using FastAPI, you will also receive a link to the API documentation. Navigate to the provided server link to access the web interface where you can submit your URL and question.

The app's frontend will render a form where you can enter the URL of the web page you want to analyze and the question you have about that page. After submitting the form, the app will display the information extracted from the page, formatted in a user-friendly manner.

Integrating the App

If you wish to integrate this app into an external service or frontend, you may need to use the server link provided by Lazy. For example, you could add the app's server link to an external tool that requires web scraping capabilities. If the app provides API endpoints, you can also integrate these endpoints into other applications or tools that you are using.

Remember, all the deployment and environment complexities are handled by Lazy, so you can focus on building and integrating your application.

If you encounter any issues or have questions about the template, refer to the Selenium documentation for more information on using Selenium with Python.

Here is an example of how you might integrate the template into another tool:


# Sample code to call the AI Scraper Selenium App API endpoint
import requests

# The URL of the page you want to scrape
page_url = "https://example.com"

# The question you want to ask about the page
question = "What is the main topic of the page?"

# The server link provided by Lazy after deployment
server_link = "http://your-lazy-app-server-link"

# The endpoint to process page information
endpoint = f"{server_link}/process-page-info"

# Prepare the data for the POST request
data = {
    "url": page_url,
    "question": question
}

# Make the POST request to the app's endpoint
response = requests.post(endpoint, data=data)

# Print the response from the app
print(response.json())

By following these steps, you can effectively use the AI Scraper Selenium App template to build and integrate a web scraping application into your projects.

More templates like this

AI Scraper Selenium App

Use this skeleton to use AI to crawl a website

Selenium App

Use this skeleton if you need to use Selenium to build the app. Example: if the user needs to write an end to end test or an automation test or a regression test or a functional test or crawl a website when it needs to do some sophisticated steps e.g. clicks, page element.

Selenium App

Use this skeleton if you need to use Selenium to build the app. Example: if the user needs to write an end to end test or an automation test or a regression test or a functional test or crawl a website when it needs to do some sophisticated steps e.g. clicks, page element.

Home

/

AI Scraper Selenium App

Terms and Conditions

Privacy Policy

Lazy AI Data Processing Agreement

Cookie Policy

Lazy AI

About us

Help Center

Subscription & Billing

Pricing

Blog

If you have any questions, please get in touch. Just or join our .