Introduction

As artificial intelligence (AI) becomes more deeply integrated into web applications, developers are increasingly handling tasks like text summarization, document parsing, language translation, and sentiment analysis. These operations, while powerful, can take several seconds—or even minutes—to complete, depending on the size and complexity of the data.

Running such heavy tasks directly during a user request can cause:

  • Long response times ⏳
  • Server timeouts ❌
  • Poor user experience 😓

The Solution: Laravel Queues

Laravel Queues offer an elegant, efficient way to offload long-running tasks to background workers. By pushing resource-heavy AI tasks into a queue, your application remains fast and responsive, while still delivering the full power of AI behind the scenes.

In this blog, we’ll walk you step-by-step through building a complete Laravel system that:

  • Accepts a document upload
  • Sends it to an AI service (e.g., summarization)
  • Processes it in the background
  • Returns the result when ready

Whether you’re a Laravel beginner or exploring AI integration for the first time, this tutorial is crafted to guide you from fundamentals to full implementation—with plenty of real-world context and practical code.

What Are Laravel Queues?

Laravel Queues are a built-in feature of the Laravel framework that allow you to defer time-consuming tasks and run them in the background using workers.

Read: Laravel Queues – A Complete Beginner’s Guide

Why Use Queues?

Normally, when a user sends a request—like uploading a document—the server processes everything immediately and responds only when it’s done. But what if:

  • The task takes 15+ seconds?
  • It depends on an external API (like OpenAI)?
  • It occasionally fails or times out?

That’s where queues come in.

How Queues Work (In Simple Terms)

  1. User triggers a task (e.g., uploading a file).
  2. Laravel stores the task in a queue (like a waiting line).
  3. A worker process picks up the task in the background.
  4. The user receives a fast response—no waiting for heavy processing.

Laravel Queue Components

  • Jobs – Pieces of work you want to queue (e.g., “Summarize this text”)
  • Queue Drivers – Where jobs are stored (e.g., database, Redis, Amazon SQS)
  • Workers – Background processes that handle jobs

This structure allows your application to remain snappy and scalable, even when processing large files or making multiple external AI requests.

:

Why Use Queues for AI Tasks?

AI-powered features—such as document summarization, natural language processing (NLP), OCR, and content classification—often rely on external APIs (like OpenAI, Hugging Face, or custom ML services). These operations can be CPU-intensive, slow, or unreliable if handled directly during the user’s request cycle.

Problems with Synchronous AI Processing

When you process AI tasks synchronously (in real time), you risk:

  • Slow response times: Large documents or API delays can freeze your UI.
  • Timeouts: Most web servers have request timeout limits (e.g., 30–60 seconds).
  • Bad UX: Users are stuck waiting without feedback.
  • High failure rates: If the API fails, it disrupts the entire request.

Benefits of Using Queues for AI Tasks

BenefitDescription
SpeedUsers get instant feedback while processing happens in the background.
RetriesFailed jobs can be retried automatically.
ScalabilityWorkers can scale separately from the app (ideal for high-load apps).
IsolationAI logic stays decoupled from user-facing code.
Timeout handlingLong-running tasks don’t impact the core app.

Real-World Examples

  • A user uploads a 10-page PDF → AI summarization happens via a queued job.
  • A user submits a job application → Resume is parsed asynchronously.
  • A support ticket is created → The system uses sentiment analysis in the background.

By combining Laravel Queues with AI services, you build an architecture that is resilient, user-friendly, and scalable.

Use Case Overview

To understand how Laravel Queues help with AI tasks, let’s define a real-world use case we’ll build in this blog:

The Scenario

A user uploads a document (e.g., .txt, .pdf, or .docx).
The app needs to:

  1. Parse the document content
  2. Send it to an AI API for summarization
  3. Store the summary
  4. Notify the user when processing is complete

Key Features We’ll Implement

  • Document Upload: Allow users to upload files.
  • Store Document in Database: Save file path and metadata.
  • Dispatch Job to Queue: Offload AI task using Laravel Jobs.
  • Process with AI API: Use HTTP client to call external AI service.
  • Save Summary: Update the document record with the AI summary.
  • Send Notification (Optional): Email or alert the user when it’s done.

Tools and Technologies

ComponentTool/Library
Backend FrameworkLaravel
Queue DriverDatabase (or Redis)
File StorageLaravel Filesystem (Local)
AI Summarization APIMock or real (e.g., OpenAI)
Job QueueLaravel Jobs & Workers
Notification SystemLaravel Notifications/Mail

By the end of this tutorial, you’ll have a fully working Laravel project that uses queues to handle AI tasks asynchronously—without slowing down your user experience.

Laravel Project Setup – Step-by-Step Guide

5.1 Laravel Project Setup

If you haven’t already created a Laravel project, follow these steps:

Prerequisites

Make sure you have:

  • PHP 8.1+
  • Composer
  • MySQL or SQLite
  • Node.js (optional, for frontend scaffolding)

Create a New Laravel Project

composer create-project laravel/laravel ai-queue-app
cd ai-queue-app

Setup Environment Configuration

Copy the .env file:

cp .env.example .env

Generate the app key:

php artisan key:generate

Update .env database config to match your local setup:

DB_DATABASE=ai_queue
DB_USERNAME=root
DB_PASSWORD=

Then run:

php artisan migrate

Run the App Locally

php artisan serve

You should see your app running at http://127.0.0.1:8000.

5.2 Configure the Queue System

Laravel supports several queue drivers, but for simplicity, we’ll use the database driver. You can switch to Redis or SQS in production.

Step 1: Update .env

Set the queue connection:

QUEUE_CONNECTION=database

Step 2: Create the Jobs Table

This will store queued jobs in the database:

php artisan queue:table
php artisan migrate

This creates a jobs table where Laravel will push queued tasks.

Step 3: Start the Queue Worker

To process jobs, run:

bashCopyEditphp artisan queue:work

Keep this terminal open. The worker listens for new jobs and processes them in the background.

You now have Laravel Queues fully configured!

5.3 Create a Document Upload Feature

This step enables users to upload files (e.g., .txt, .pdf, .docx) which we’ll later pass to the AI summarization job.

Step 1: Create the Document Model and Migration

Run the following Artisan command:

php artisan make:model Document -m

This creates a Document.php model and a migration file.

Step 2: Define the Table Structure

Open the migration file located in database/migrations/xxxx_xx_xx_create_documents_table.php and update it:

public function up()
{
    Schema::create('documents', function (Blueprint $table) {
        $table->id();
        $table->string('file_path');
        $table->text('summary')->nullable();
        $table->string('status')->default('pending');
        $table->timestamps();
    });
}

Run the migration:

php artisan migrate

Step 3: Create Upload Route and Controller

Add a route in routes/web.php (or routes/api.php if using API routes):

use App\Http\Controllers\DocumentController;

Route::post('/upload', [DocumentController::class, 'upload']);

Create the controller:

php artisan make:controller DocumentController

Step 4: Implement Upload Logic

In app/Http/Controllers/DocumentController.php:

namespace App\Http\Controllers;

use App\Models\Document;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Storage;

class DocumentController extends Controller
{
    public function upload(Request $request)
    {
        $request->validate([
            'document' => 'required|file|mimes:txt,pdf,doc,docx|max:5120', // Max 5MB
        ]);

        $path = $request->file('document')->store('documents');

        $document = Document::create([
            'file_path' => $path,
            'status' => 'pending',
        ]);

        return response()->json([
            'message' => 'Document uploaded successfully.',
            'document_id' => $document->id,
        ]);
    }
}

Step 5: Storage Setup (If Needed)

If you’re using local storage and want the uploaded documents accessible:

php artisan storage:link

This creates a symbolic link from public/storage to storage/app/public.

5.4 Create the Queue Job for AI Tasks

This step is where we define what happens in the background after a document is uploaded—such as reading the content, calling an AI API for summarization, and saving the response.

Step 1: Generate the Job

Run the Artisan command:

bashCopyEditphp artisan make:job ProcessAIDocument

This creates a job file in app/Jobs/ProcessAIDocument.php.

Step 2: Modify the Job Logic

Open ProcessAIDocument.php and update it with the logic to handle AI processing:

namespace App\Jobs;

use App\Models\Document;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Storage;

class ProcessAIDocument implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    protected $document;

    /**
     * Create a new job instance.
     */
    public function __construct(Document $document)
    {
        $this->document = $document;
    }

    /**
     * Execute the job.
     */
    public function handle()
    {
        // 1. Read the file content
        $filePath = storage_path('app/' . $this->document->file_path);
        $content = file_get_contents($filePath);

        // 2. Call AI API (e.g., OpenAI, HuggingFace, custom)
        // This is just a dummy request — replace with your real AI API
        $response = Http::post('https://api.example.com/summarize', [
            'text' => $content,
        ]);

        // 3. Extract the summary
        $summary = $response->json()['summary'] ?? 'Summary not available.';

        // 4. Update the document record
        $this->document->update([
            'summary' => $summary,
            'status' => 'processed',
        ]);
    }
}

Notes:

  • Queueable Payload: Since Eloquent models can’t be serialized fully, Laravel automatically fetches fresh model data when the job runs.
  • AI API URL: Replace https://api.example.com/summarize with your actual AI service endpoint (e.g., OpenAI’s /v1/completions or /v1/chat/completions).
  • Error Handling: You can enhance the job by wrapping the API call in a try-catch block and logging failures.

5.5 Dispatch the Job

Once the user uploads a document, we want to immediately queue a job to process it using the AI API in the background.

Step 1: Import the Job Class

In DocumentController.php, import the job at the top:

use App\Jobs\ProcessAIDocument;

Step 2: Dispatch the Job After Upload

Update the upload() method in your DocumentController:

public function upload(Request $request)
{
    $request->validate([
        'document' => 'required|file|mimes:txt,pdf,doc,docx|max:5120',
    ]);

    $path = $request->file('document')->store('documents');

    $document = Document::create([
        'file_path' => $path,
        'status' => 'pending',
    ]);

    // Dispatch the background job
    ProcessAIDocument::dispatch($document);

    return response()->json([
        'message' => 'Document uploaded. Processing will continue in background.',
        'document_id' => $document->id,
    ]);
}

Step 3: Run the Queue Worker

Make sure a worker is running to process jobs:

php artisan queue:work

Leave this running in a separate terminal window. Every time a user uploads a document, Laravel will:

  1. Store the file
  2. Add the AI processing job to the queue
  3. The worker will pick it up and process it in the background

That’s it! You’ve now successfully dispatched your AI task into a queue—completely separated from the user’s request.

5.6 Process the Job and Store AI Response

In step 5.4, we already created the ProcessAIDocument job with basic logic. Now, let’s enhance it for reliability, real AI integration, and error handling.

Step 1: Ensure You Have File Access

We need to access the uploaded file content. Laravel stores files in storage/app/ by default, so we read from there.

$filePath = storage_path('app/' . $this->document->file_path);
$content = file_get_contents($filePath);

Make sure:

  • File path is correct
  • File format is readable (e.g., .txt)
  • You add proper fallbacks for unreadable files

Step 2: Connect to an AI API (Example: OpenAI)

Let’s assume you’re using OpenAI’s summarization model via their API:

$response = Http::withHeaders([
    'Authorization' => 'Bearer ' . config('services.openai.key'),
])->post('https://api.openai.com/v1/chat/completions', [
    'model' => 'gpt-3.5-turbo',
    'messages' => [
        ['role' => 'system', 'content' => 'You are a document summarizer.'],
        ['role' => 'user', 'content' => $content],
    ],
    'temperature' => 0.7,
    'max_tokens' => 500,
]);

Extract the response safely:

$summary = $response->json()['choices'][0]['message']['content'] ?? 'AI failed to summarize.';

Step 3: Add Error Handling and Logging

Enhance your handle() method with error management:

public function handle()
{
    try {
        $filePath = storage_path('app/' . $this->document->file_path);
        $content = file_get_contents($filePath);

        if (!$content) {
            throw new \Exception("Empty or unreadable document content.");
        }

        $response = Http::withHeaders([
            'Authorization' => 'Bearer ' . config('services.openai.key'),
        ])->post('https://api.openai.com/v1/chat/completions', [
            'model' => 'gpt-3.5-turbo',
            'messages' => [
                ['role' => 'system', 'content' => 'You are a document summarizer.'],
                ['role' => 'user', 'content' => $content],
            ],
            'temperature' => 0.7,
            'max_tokens' => 500,
        ]);

        $summary = $response->json()['choices'][0]['message']['content'] ?? 'Summary not available.';

        $this->document->update([
            'summary' => $summary,
            'status' => 'processed',
        ]);

    } catch (\Throwable $e) {
        // Log and mark document as failed
        \Log::error('AI job failed: ' . $e->getMessage());

        $this->document->update([
            'summary' => 'An error occurred during summarization.',
            'status' => 'failed',
        ]);
    }
}

Optional: Add Job Timeout and Retries

You can set this in your job class:

public $timeout = 120; // Max 2 minutes per job
public $tries = 3;     // Retry up to 3 times on failure

5.7 Notify the User When Done (Optional)

Notifying users when their document has been summarized enhances UX—especially for long or asynchronous tasks. Laravel makes this easy with its built-in notification system.

Step 1: Add a Relationship Between Document and User

Make sure your Document model has a link to a user.

Update Migration

In your documents migration file or a new one:

$table->foreignId('user_id')->constrained()->onDelete('cascade');

Then update your model:

// app/Models/Document.php
public function user()
{
    return $this->belongsTo(User::class);
}

Step 2: Create a Notification Class

Generate a new notification:

php artisan make:notification DocumentProcessed

Open app/Notifications/DocumentProcessed.php and configure it:

use Illuminate\Notifications\Notification;
use Illuminate\Notifications\Messages\MailMessage;

class DocumentProcessed extends Notification
{
    protected $document;

    public function __construct($document)
    {
        $this->document = $document;
    }

    public function via($notifiable)
    {
        return ['mail']; // You can add 'database' or 'broadcast' here too
    }

    public function toMail($notifiable)
    {
        return (new MailMessage)
            ->subject('Your Document Summary is Ready')
            ->greeting('Hello!')
            ->line('Your document has been processed and summarized.')
            ->line('Summary:')
            ->line($this->document->summary)
            ->action('View Document', url('/documents/' . $this->document->id))
            ->line('Thank you for using our AI service!');
    }
}

Step 3: Send Notification from the Job

In ProcessAIDocument.php, update the handle() method (inside success block):

$this->document->update([
    'summary' => $summary,
    'status' => 'processed',
]);

// Notify the user
$this->document->user->notify(new DocumentProcessed($this->document));

Make sure your User model uses the Notifiable trait:

use Illuminate\Notifications\Notifiable;

class User extends Authenticatable
{
    use Notifiable;
    ...
}

Step 4: Configure Mail Settings

In your .env file:

MAIL_MAILER=smtp
MAIL_HOST=smtp.mailtrap.io
MAIL_PORT=2525
MAIL_USERNAME=your_username
MAIL_PASSWORD=your_password
MAIL_ENCRYPTION=null
MAIL_FROM_ADDRESS=hello@example.com
MAIL_FROM_NAME="AI Queue App"

Test with a tool like Mailtrap, MailHog, or a real SMTP server.

6. Running the Queue Worker

Once your Laravel application is dispatching jobs (like summarizing documents), those jobs sit in a queue (e.g., in the database) until a worker process picks them up and executes them.

Laravel provides a command-line worker to handle this for you.

Step 1: Start the Worker

Open a new terminal and run:

php artisan queue:work

This starts the queue worker which:

  • Listens for new jobs
  • Processes them as they come in
  • Outputs logs/errors in real time

Keep this process running in the background. If you stop it, queued jobs won’t be processed.

Optional: Run With Logging

To keep logs of processed jobs:

bashCopyEditphp artisan queue:work --tries=3 --timeout=120 >> storage/logs/queue.log

Step 2: Keep Worker Running in Production

On a production server, you should not run workers manually in the terminal. Instead, use a process manager like:

A. Supervisor (Linux Servers)

Install Supervisor and configure it to keep queue:work running 24/7.

Example config (/etc/supervisor/conf.d/laravel-worker.conf):

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /path/to/your/app/artisan queue:work --sleep=3 --tries=3
autostart=true
autorestart=true
user=your_user
numprocs=1
redirect_stderr=true
stdout_logfile=/path/to/your/app/storage/logs/laravel-worker.log

Then reload:

sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start laravel-worker:*

📊 B. Laravel Horizon (for Redis Queues)

If you’re using Redis, Laravel Horizon gives you a beautiful dashboard and robust monitoring tools.

Bonus: Using Laravel Horizon for Monitoring

Laravel Horizon is an elegant dashboard and code-based configuration system for your Laravel queues. It works exclusively with Redis and gives you real-time insights into:

  • Job processing status
  • Failures and retries
  • Queue throughput
  • Job history
  • Worker status and metrics

Step 1: Install Horizon

Install the Horizon package via Composer:

composer require laravel/horizon

Step 2: Publish Assets and Config

Publish Horizon’s service provider and assets:

php artisan horizon:install

Run the required migration:

php artisan migrate

This creates the necessary tables to track metrics.

Step 3: Use Redis as Your Queue Driver

In .env:

envCopyEditQUEUE_CONNECTION=redis

Also make sure you have Redis installed and running locally or via Docker.

Step 4: Run Horizon

Start Horizon in your terminal:

php artisan horizon

This launches Horizon’s queue worker and dashboard engine.

Unlike queue:work, Horizon includes auto-balancing, process management, and load monitoring.

Step 5: Open the Horizon Dashboard

Visit:

http://localhost/horizon

You’ll see:

  • Active jobs in real time
  • Failed jobs with full error stack trace
  • Queue size trends
  • Processing durations
  • Throughput per minute

It’s a powerful visual tool for production-grade apps.

Step 6: Monitor Horizon in Production

In production, you can use Supervisor to run Horizon persistently:

[program:horizon]
process_name=%(program_name)s
command=php /path/to/your/project/artisan horizon
autostart=true
autorestart=true
user=your_user
redirect_stderr=true
stdout_logfile=/path/to/your/project/storage/logs/horizon.log

8. Common Errors and Debugging Tips

When working with Laravel Queues and AI integrations, it’s normal to run into issues—especially around serialization, API timeouts, or worker configuration. Here’s how to troubleshoot the most common problems:

1. Jobs Not Processing

Symptoms:

  • Job enters the queue (jobs table) but nothing happens.

Fixes:

  • Ensure you’re running a worker: bashCopyEditphp artisan queue:work
  • Check queue connection in .env: envCopyEditQUEUE_CONNECTION=database // or redis
  • Look for syntax errors or missing class references in your Job class.

2. Job Retries or Infinite Loop

Symptoms:

  • Job keeps retrying
  • Worker restarts repeatedly

Fixes:

  • Set tries and timeout values in your Job class: phpCopyEditpublic $tries = 3; public $timeout = 120;
  • Wrap handle() logic in a try-catch block to handle exceptions gracefully.

3. Serialization Errors

Symptoms:

  • Error like: Serialization of 'Closure' is not allowed

Fixes:

  • Do not pass closures or anonymous functions to jobs.
  • Use models or basic data types (arrays, strings, integers) only.

✅ Good:

phpCopyEditProcessAIDocument::dispatch($document);

🚫 Bad:

phpCopyEditProcessAIDocument::dispatch(function () { ... });

4. AI API Timeout or Failure

Symptoms:

  • Job fails on external API call
  • Error: “cURL error 28: Operation timed out”

Fixes:

  • Add timeout to your HTTP call: phpCopyEdit$response = Http::timeout(30)->post(...);
  • Use Laravel’s retry and fail logic to avoid crashes.

5. Emails Not Sending

Symptoms:

  • No notification received
  • No error in logs

Fixes:

  • Confirm mail settings in .env
  • Test mail using Mail::raw() or tinker
  • Use Mailtrap, MailHog, or a working SMTP server in local

6. Debugging Tools

  • View failed jobs: php artisan queue:failed
  • Retry a failed job: php artisan queue:retry <job-id>
  • Remove failed jobs: php artisan queue:flush
  • View logs: tail -f storage/logs/laravel.log

9. Conclusion

Integrating AI features like document summarization, parsing, or NLP into your Laravel application can deliver powerful user experiences—but these tasks are often slow, resource-intensive, and prone to failure if done synchronously.

By offloading AI tasks to Laravel Queues, you’ve built a system that is:

  • Fast – Keeps user experience snappy by processing tasks in the background
  • Reliable – Automatically retries failed jobs and isolates long processes
  • Scalable – Works well even as your user or document load increases
  • AI-ready – Prepared to plug into modern AI APIs like OpenAI, Claude, or custom models

What You Built

You’ve successfully created a Laravel-based system that:

  1. Accepts file uploads from users
  2. Stores documents and metadata
  3. Queues AI processing tasks using jobs
  4. Sends those tasks to an external AI API
  5. Stores the AI-generated summary in the database
  6. (Optionally) notifies users when the task is complete
  7. Uses Laravel Horizon to monitor all activity in real-time

What’s Next?

  • Integrate OCR (for scanned PDFs)
  • Add a front-end dashboard to track job status
  • Implement user-based rate limiting
  • Add support for additional languages
  • Use Redis and deploy Horizon to production

Missed the Basics?

If you’re unfamiliar with how Laravel Queues work under the hood, we recommend reading this first:

Laravel Queues – A Complete Beginner’s Guide

Final Thoughts

By combining Laravel’s robust queue system with modern AI services, you’ve unlocked the potential to build highly responsive, intelligent, and scalable applications.

Thanks for following along—now go build something amazing!

Categorized in:

AILaravel

Tagged in:

, ,