Guides
Guides

Getting Started with Data Ingestion

Get started with Nacelle by ingesting your content, collections, products and pricing data

Suggest Edits

To get started with Nacelle, start with Data Ingestion. Data ingestion is the process of collecting, importing, and processing data from various sources into Nacelles storage platform where it can be analyzed, processed, and used to build your commerce platform. Data ingestion is a required step in the data pipeline, because it ensures that relevant data is available and accessible for further processing and building, ultimately enabling organizations to build their commerce platform.

Getting started

Follow these steps to ingest your data into Nacelle:

  1. Complete the prerequisites, including creating an account, accessing your dashboard, and deciding whether to use the REST API or the pre-built connectors.
  2. Get your Ingest API request headers.
  3. Start an index, and then finish the index to promote it to be your active index.
  4. Add content, collections, products and pricing data to your index.

📘

Note: Use ingestion to import your product, pricing and collections data into Nacelle. If you are looking to send data back and forth between Nacelle and your commerce tools, check out the Mapper docs.

Before you begin

Be sure to complete these steps before trying to ingest your data into Nacelle.

  1. Create your Nacelle account
    Get started by signing up for an account.
  2. Access your Nacelle dashboard
    Once you've signed up for an account and confirmed your email, you'll gain access to your Nacelle dashboard. Here, you'll find resources and tools to help you set up and manage your data and storefront.
  3. Decide whether to use the Ingestion REST API or the Pre-built Connectors

Evaluate your specific requirements, data sources, and integration needs to determine the most suitable approach for efficiently ingesting data into Nacelle:

  • Pre-built Connectors: If your data originates from commonly used platforms like Contentful, Sanity, or Shopify, leveraging the pre-built connectors can streamline the ingestion process, saving time and effort.
  • Ingestion REST API: If your data sources are more diverse or custom, or if you overall want more control of your data, using the Ingestion REST API provides greater flexibility and control over how data is imported and processed into Nacelle's system.

✅

Not sure the best way to ingest your data? Contact support to discuss your specific needs.

Ingesting your data

This guide explains how to ingest your data using the Ingestion REST API. If you want to use the pre-built connectors, see their specific pages: Contentful, Sanity, and Shopify.

Authentication

  1. Set up a custom connector in the Nacelle dashboard. Setting up the custom connector gives you a x-nacelle-source-id to use in the REST API header.
  2. Next, find your x-nacelle-space-id. To find the Space ID, login to your Nacelle Dashboard > select your space > click Space Settings > click the API Details tab > find the x-nacelle-space-id in the Space ID section.
  3. Last, generate a x-nacelle-ingest-token. To create an Ingest Token, login to your Nacelle Dashboard > select your space > click Space Settings > click the API Details tab > click the Create a New Key button under Private Token in the Data Ingest API section. The key will only be displayed once, so be sure to save it in a secure location

Starting an index

To start an index, use the POST https://ingest.api.nacelle.com/v1/index endpoint. In an entryTypes JSON Schema, define the type of index you want to create. This is where you evaluate the type of data that you are going to include in the indices by looking at what you plan to do with that data. Valid values include:

  • COLLECTION: Specifies a grouping or category of related items or products.
    • COLLECTION_CONTENT: Content associated with a specific collection, such as descriptions or images.
  • CONTENT: Content and copy ingested from a CMS.
  • PRICE_BOOK: Defines a set of prices for products, typically used for different pricing strategies or customer segments.
  • PRICING: Pricing information associated with products or variants.
  • PRODUCT: Individual items available for sale.
    • PRODUCT_CONTENT: Content specific to a particular product, including details, descriptions, and media.
  • PRODUCT_VARIANT: Variations of a product, such as different sizes or colors.
    • PRODUCT_VARIANT_CONTENT: The content specific to a product variant, such as variant-specific details or images.
  • CUSTOM_SCHEMA: Allows for the ingestion of data structured according to a custom-defined schema, enabling flexibility in data organization and representation.

Example

Uses the schema from Start an Index.

Request

curl --request POST \
     --url https://ingest.api.nacelle.com/v1/index \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --header 'x-nacelle-ingest-token: {your ingest token}' \
     --header 'x-nacelle-source-id: {your source ID}' \
     --header 'x-nacelle-space-id: {your space ID}' \
     --data '
{
  "entryTypes": [
    "PRODUCT",
    "COLLECTION",
    "PRODUCT_VARIANT"
  ]
}

Response

{
  "message": "Index Created",
  "indexId": "{your new index ID}"
}

Finishing an Index

You can only have one active index at a time. To make an index your active index, use the POST Finish an Index endpoint.

Example

Uses the schema from Finish an Index.

Request

curl --request POST \
     --url https://ingest.api.nacelle.com/v1/index/b9367bc5-ac9c-4409-9016-fb23f8819314/finish \
     --header 'accept: application/json' \
     --header 'x-nacelle-ingest-token: {your ingest token}' \
     --header 'x-nacelle-source-id: {your source ID}' \
     --header 'x-nacelle-space-id: {your space ID}' \

Response

{
  "message": "Index Finished"
}

Adding content, collections, products, and pricing data

Nacelle's data ingestion system uses a hierarchical structure to manage content, collections, products, and pricing data. This system ensures that your commerce data is organized and easily accessible. Now that you've created an index, you can begin adding your content, collections, products and pricing data to it. The logic is that you use separate endpoints to add top-level entries such as content, collections, products, and pricing data to the index. Most of these top-level entries (except content) can then have additional content added to them through separate endpoints. You can update the content of these entries at any time to keep your data current and relevant.

  • Product - Endpoint to add product entries
    • Product Content - Endpoint to add product content
    • Variant - Endpoint to add product variants. For example, sizes, colors or types.
  • Collection - Endpoint to add a collection of products. For example, all shoes, or all bags.
  • Content - Endpoint to add CMS content to your index.

Example

Uses the schema from Add a Product, and Add Product Content.

Request

Add a Product

curl --request PUT \
     --url https://ingest.api.nacelle.com/v1/product \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --header 'x-nacelle-ingest-token: {your ingest token}' \
     --header 'x-nacelle-source-id: {your source ID}' \
     --header 'x-nacelle-space-id: {your space ID}' \
     --data '
{
  "indexId": "e8a52947-42f8-439d-bf02-98098d32014f",
  "entries": [
    {
      "taxes": {
        "taxProvider": "Avalara"
      },
      "sourceEntryId": "gid://shopify/Product/0000000000000",
      "availableForSale": false
    }
  ]
}

Response

{"message":"PRODUCT UPDATE: success"}

Add Product Content

curl --request PUT \
     --url https://ingest.api.nacelle.com/v1/product-content \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --header 'x-nacelle-ingest-token: {your ingest token}' \
     --header 'x-nacelle-source-id: {your source ID}' \
     --header 'x-nacelle-space-id: {your space ID}' \
     --data '@product.json'

Example product.json

{
  "entries": [
    {
      "featuredMedia": {
        "src": "https://images.salsify.com/image/upload/s--8qOuPa0f--/3bff7c48b0ade02ff35783f29ac08352de8b44cd.jpg",
        "thumbnailSrc": "https://images.salsify.com/image/upload/s--8qOuPa0f--/3bff7c48b0ade02ff35783f29ac08352de8b44cd.jpg",
        "type": "IMAGE"
      },
      "sourceProductId": "gid://shopify/Product/0000000000000",
      "sourceEntryId": "3858f62230ac3c915f300c664312c63f",
      "title": "Wayfarer Messenger Bag",
      "description": "<p>Perfect for class, work or the gym, the Wayfarer Messenger Bag is packed with pockets. The dual-buckle flap closure reveals an organizational panel, and the roomy main compartment has spaces for your laptop and a change of clothes. An adjustable shoulder strap and easy-grip handle promise easy carrying.</p>\\n<ul>\\n<li>Multiple internal zip pockets.</li>\\n<li>Made of durable nylon.</li>\\n</ul>",
      "media": [
        {
          "src": "https://images.salsify.com/image/upload/s--8qOuPa0f--/3bff7c48b0ade02ff35783f29ac08352de8b44cd.jpg",
          "thumbnailSrc": "https://images.salsify.com/image/upload/s--8qOuPa0f--/3bff7c48b0ade02ff35783f29ac08352de8b44cd.jpg",
          "type": "IMAGE"
        }
      ],
      "metafields": [
        {
          "key": "Brand",
          "value": "Luma",
          "namespace": "filter"
        },
        {
          "key": "vipPriceFactor",
          "value": "0.9",
          "namespace": "segmentPrice"
        }
      ]
    }
  ]
}

Response

{
     "message": "PRODUCT_CONTENT UPDATE: success"
   }

Support

If you get stuck, send us an email or use the Intercom widget on the bottom right of your Nacelle Dashboard.

What's next

Updated 7 months ago