AI Web scraper

 import logging
from fastapi import FastAPI, Form, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
# Removed WebFetcher import as it's no longer needed with the new Google CSE integration
import uvicorn
from typing import List
from abilities import llm_prompt

app = FastAPI()
templates = Jinja2Templates(directory="templates")

# Removed SolutionFinder class as it's no longer needed with the new Google CSE integration

import json
@app.get("/", response_class=HTMLResponse)
async def read_index(request: Request):
    return templates.TemplateResponse("index.html", {"request": request})

@app.get("/solutions", response_class=HTMLResponse)
async def solutions(request: Request):
    solutions =

About this template

AI Web Scraper A web app that uses google to generate a curated list of websites that can help solve specific problems or situations.

Introduction to the AI Web Scraper Template

Welcome to the AI Web Scraper template! This template is designed to help you build a web application that leverages Google's Custom Search Engine to generate a curated list of websites. These websites can provide solutions to specific problems or situations you may encounter. This step-by-step guide will walk you through the process of using this template on the Lazy platform, ensuring you can deploy your application with ease.

Getting Started

To begin using the AI Web Scraper template, simply click on "Start with this Template" within the Lazy Builder interface. This will pre-populate the code in the Lazy Builder, so you won't need to copy, paste, or delete any code manually.

Test: Deploying the App

Once you have started with the template, the next step is to deploy your application. Press the "Test" button to initiate the deployment process. The Lazy CLI will handle the deployment, and you won't need to worry about installing libraries or setting up your environment.

Entering Input

If the template requires user input, the Lazy App's CLI interface will prompt you for it after you press the "Test" button. Provide the necessary input as prompted to ensure the app functions correctly.

Using the App

After deployment, if the template includes a frontend, you will be able to interact with the user interface. The AI Web Scraper app provides an interface where you can search for solutions using Google's Custom Search Engine. You can access this interface through a dedicated server link that Lazy will print out for you.

If the code uses FastAPI, Lazy will also provide you with a documentation link, which you can use to understand and interact with the API endpoints.

Integrating the App

If you need to integrate the AI Web Scraper app into an external service or frontend, you may need to add the app's server link or API endpoints to the external tool. Follow the specific instructions provided by the external tool to complete this integration.

For example, if you need to add API endpoints to an external tool, ensure you have the correct scopes and know where to add the code. If the template includes sample code that is helpful for integration, make sure to use it as a reference.

Remember, this guide is here to help you through the mandatory steps to run and integrate the AI Web Scraper template. If you encounter any issues or have questions, refer to the documentation links provided in the code or reach out for support through the Lazy platform.

Happy building!

Last published
April 7, 2024

More templates like this

Customisable Server Status Dashboard

This template allows you to create a dashboard to report about incidents regarding your server to your users.


A bot that answers questions about data

Ask questions about your database via a chat bot. This chatbot connects to a SQLITE database, generates queries for it based on the schema and then runs the queries printing the response all in a nicely styled chat UI. This template is pre-loaded with sample data (car parts) to play around with - here are the sample data columns that you can query: part_number, part_name, price, units_left_in_stock, manufacturer .


WhatsApp Bot Builder

The WhatsApp Bot Builder app allows users to create a WhatsApp bot that responds to every message with "Hello World".

AI Web scraper