Building an Advanced AI Workflow Automation with Next.js and DataSpeck

What We’re Building

In this tutorial, we will build an advanced AI-driven workflow automation system. We will use DataSpeck—a cutting-edge system introduced at CHI 2026 that leverages an AI-driven human-in-the-loop architecture to generate data transformation code dynamically. This solves the notoriously difficult problem of semantic ambiguities in large datasets.

Our finished project will be a Next.js web application that allows users to define raw data conversion tasks. The Next.js backend will securely pass this to DataSpeck to generate and execute the necessary Python transformation scripts, while surfacing a frontend UI for human review and refinement.

Note: As DataSpeck recently debuted as a research prototype, the API endpoint URLs in this tutorial are illustrative. Please adjust the URLs according to your specific local deployment or enterprise access portal.

Setup and Installation

First, bootstrap a modern Next.js environment and install the necessary HTTP client for our server-side communications.

npx create-next-app@latest dataspeck-automation
cd dataspeck-automation
npm install axios

Securely configure your environment variables. Never expose your DataSpeck API key to the client browser. Create a .env.local file in your root directory:

# .env.local
DATASPECK_API_KEY=sk-ds-your_secure_api_key_here

Step 1: The Client-Side UI (React Server Components)

Because we are using the Next.js App Router, components default to Server Components. To handle user state and button clicks, we must explicitly declare our interface as a Client Component using the "use client" directive.

'use client';

import { useState } from 'react';
import axios from 'axios';

export default function DataConversionInterface() {
  const [inputData, setInputData] = useState('');
  const [conversionResult, setConversionResult] = useState(null);
  const [isLoading, setIsLoading] = useState(false);

  const handleConvert = async () => {
    setIsLoading(true);
    try {
      // Calls our secure Next.js Route Handler, not the external API directly
      const response = await axios.post('/api/convert', { data: inputData });
      setConversionResult(response.data);
    } catch (error) {
      console.error('Error converting data:', error);
    } finally {
      setIsLoading(false);
    }
  };

  return (
    <div className="p-6 max-w-2xl mx-auto">
      <h1 className="text-2xl font-bold mb-4">DataSpeck Conversion Hub</h1>
      <textarea
        className="w-full p-4 border rounded"
        rows="6"
        value={inputData}
        onChange={(e) => setInputData(e.target.value)}
        placeholder="Paste your raw CSV or JSON array here..."
      />
      <button 
        className="mt-4 px-6 py-2 bg-blue-600 text-white rounded"
        onClick={handleConvert} 
        disabled={isLoading}
      >
        {isLoading ? 'Processing via AI...' : 'Generate Transformation'}
      </button>
      
      {conversionResult && (
        <div className="mt-6 p-4 bg-gray-50 rounded">
           <h3 className="font-semibold">AI Output:</h3>
           <pre className="text-sm mt-2">{JSON.stringify(conversionResult, null, 2)}</pre>
        </div>
      )}
    </div>
  );
}
  

Step 2: Creating the Secure Route Handler

Next, we build the API Route Handler. In Next.js 14+, this lives in app/api/convert/route.js. This server-side function securely communicates with DataSpeck, keeping your API keys hidden from the browser.

import { NextResponse } from 'next/server';
import axios from 'axios';

export async function POST(request) {
  try {
    const { data } = await request.json();
    const apiKey = process.env.DATASPECK_API_KEY;
    const apiUrl = 'https://api.dataspeck.com/v1/transform'; // Adjust to your instance

    if (!data) {
      return NextResponse.json({ error: 'Data payload is required' }, { status: 400 });
    }

    const response = await axios.post(apiUrl, { payload: data }, {
      headers: {
        'Authorization': `Bearer ${apiKey}`,
        'Content-Type': 'application/json'
      }
    });
    
    return NextResponse.json(response.data);
  } catch (error) {
    console.error('DataSpeck API Error:', error.response?.data || error.message);
    return NextResponse.json({ error: 'Failed to process data conversion' }, { status: 500 });
  }
}
  

Step 3: Integrating the Human-in-the-Loop Feedback

DataSpeck shines when dealing with semantic ambiguities (e.g., deciding if “St.” means “Street” or “Saint”). We need a component to capture human review before the final code execution is approved.

'use client';

import { useState } from 'react';
import axios from 'axios';

export default function HumanReviewLayer({ reviewData }) {
  const [feedback, setFeedback] = useState('');
  const [status, setStatus] = useState('pending');

  const handleSubmitFeedback = async () => {
    setStatus('submitting');
    try {
      await axios.post('/api/feedback', { 
        jobId: reviewData.jobId, 
        userCorrection: feedback 
      });
      setStatus('resolved');
      alert('Feedback integrated. Re-running transformation script!');
    } catch (error) {
      console.error('Feedback sync failed:', error);
      setStatus('error');
    }
  };

  if (!reviewData) return null;

  return (
    <div className="border-t-2 mt-8 pt-6">
      <h2 className="text-xl font-bold text-red-600">Human Review Required</h2>
      <p className="mb-4 text-gray-600">The AI detected an ambiguity in the mapping logic.</p>
      
      <textarea
        className="w-full p-3 border rounded border-red-300"
        value={feedback}
        onChange={(e) => setFeedback(e.target.value)}
        placeholder="Provide semantic context (e.g., 'Convert all dates to ISO 8601')"
      />
      <button 
        className="mt-3 px-4 py-2 bg-green-600 text-white rounded"
        onClick={handleSubmitFeedback}
      >
        {status === 'submitting' ? 'Syncing...' : 'Submit Human Override'}
      </button>
    </div>
  );
}
  

Testing Your Implementation

To verify the workflow, boot up your Next.js development server and pass an intentionally ambiguous dataset to trigger the human-in-the-loop review.

# Start Next.js App Router
npm run dev

# Example payload to trigger ambiguity
[
  { "customer": "J. Doe", "address": "123 Main St." },
  { "customer": "A. Smith", "address": "456 St. John Ave" }
]

What to Build Next

• Webhooks: Implement server-side Webhooks to listen for asynchronous completion events when DataSpeck finishes generating large Python scripts.
• Version Control: Log the generated transformation scripts into an AWS S3 bucket to create an audit trail of the AI’s logic over time.

🤖 Interactive Developer Support

Implementing Human-in-the-Loop architectures with Next.js App Router involves managing complex asynchronous states and secure API routing. You don’t have to troubleshoot alone.

Are you struggling to connect the Route Handlers or facing formatting issues with DataSpeck?

Engage with the Gate of AI Technical Assistant directly in the chat interface below. Our interactive AI is fully context-aware of this tutorial and can help you debug your React components, optimize your Next.js API routes, or refine your AI feedback loop.