Skip to main content

​
Overview

This guide demonstrates how to scrape data from websites that require user authentication by using persistent browser sessions. You’ll learn to launch a browser session to create a browser profile, authenticate manually, and then run automated tasks using that authenticated profile.
1

Launch Browser Session

First, open a browser session to create a new browser profile that will persist your authentication cookies. This session will return a profile_id and live_url for manual authentication.
Python
from smooth import SmoothClient

smooth_client = SmoothClient(api_key="cmzr-YOUR_API_KEY")
session = smooth_client.open_session(profile_id="gmail-session")

print(f"Live URL: {session.live_url()}")
print(f"Session ID: {session.profile_id()}")
The live_url opens an interactive browser window where you can manually authenticate.
2

Authenticate Manually

Open the live_url in your browser and manually log in to Gmail:
  1. Navigate to the live URL provided by the session
  2. Go to gmail.com in the browser
  3. Enter your email address and password
  4. Complete any two-factor authentication if required
  5. Verify you’re successfully logged in to your Gmail inbox
Your authentication cookies are now securely stored in the browser profile and will persist for future tasks.
3

Close Browser Session

Use the returned live_id to close the interactive session and save the browser profile
Python
smooth_client.close_session(live_id=session.live_id())
If the browser session is not manually closed, the profile is saved when the 5 minutes time limit is reached and the browser session closed automatically.
4

Run Automated Task

Now use the authenticated profile to run an automated task. The agent will access Gmail as a logged-in user and retrieve your last email.
Python
task = smooth_client.run(
    task="Go to Gmail and get the subject and sender of the most recent email in my inbox",
    profile_id="gmail-session"
)

print(f"Live URL: {task.live_url()}")
result = task.result()
print(f"Last email details: {result}")
The agent will automatically use the stored authentication from your profile to access Gmail and extract the requested information.

​
Key Benefits

  • Persistent Authentication: Profiles maintain login state across multiple tasks
  • Manual Control: You handle the authentication process manually for security
  • Automated Execution: Once authenticated, agents can run complex tasks automatically
  • Session Reuse: The same profile can be used for multiple related tasks

​
Best Practices

  • Use descriptive profile IDs for better organization
  • Keep profiles secure and don’t share profile IDs
  • Test authentication manually before running automated tasks
  • Handle rate limits and be respectful to the target website
Browser profiles persist cookies and authentication state, making them perfect for accessing protected content while maintaining security through manual authentication.

​
Community

Join Discord

Join our community for support and showcases