Getting Started with Data Ingestion
Get started with Nacelle by ingesting your content, collections, products and pricing data
To get started with Nacelle, start with Data Ingestion. Data ingestion is the process of collecting, importing, and processing data from various sources into Nacelles storage platform where it can be analyzed, processed, and used to build your commerce platform. Data ingestion is a required step in the data pipeline, because it ensures that relevant data is available and accessible for further processing and building, ultimately enabling organizations to build their commerce platform.
Getting started
Follow these steps to ingest your data into Nacelle:
- Complete the prerequisites, including creating an account, accessing your dashboard, and deciding whether to use the REST API or the pre-built connectors.
- Get your Ingest API request headers.
- Start an index, and then finish the index to promote it to be your active index.
- Add content, collections, products and pricing data to your index.
Note: Use ingestion to import your product, pricing and collections data into Nacelle. If you are looking to send data back and forth between Nacelle and your commerce tools, check out the Mapper docs.
Before you begin
Be sure to complete these steps before trying to ingest your data into Nacelle.
- Create your Nacelle account
Get started by signing up for an account. - Access your Nacelle dashboard
Once you've signed up for an account and confirmed your email, you'll gain access to your Nacelle dashboard. Here, you'll find resources and tools to help you set up and manage your data and storefront. - Decide whether to use the Ingestion REST API or the Pre-built Connectors
Evaluate your specific requirements, data sources, and integration needs to determine the most suitable approach for efficiently ingesting data into Nacelle:
- Pre-built Connectors: If your data originates from commonly used platforms like Contentful, Sanity, or Shopify, leveraging the pre-built connectors can streamline the ingestion process, saving time and effort.
- Ingestion REST API: If your data sources are more diverse or custom, or if you overall want more control of your data, using the Ingestion REST API provides greater flexibility and control over how data is imported and processed into Nacelle's system.
Not sure the best way to ingest your data? Contact support to discuss your specific needs.
Ingesting your data
This guide explains how to ingest your data using the Ingestion REST API. If you want to use the pre-built connectors, see their specific pages: Contentful, Sanity, and Shopify.
Authentication
- Set up a custom connector in the Nacelle dashboard. Setting up the custom connector gives you a
x-nacelle-source-id
to use in the REST API header. - Next, find your
x-nacelle-space-id
. To find the Space ID, login to your Nacelle Dashboard > select your space > click Space Settings > click the API Details tab > find thex-nacelle-space-id
in the Space ID section. - Last, generate a
x-nacelle-ingest-token
. To create an Ingest Token, login to your Nacelle Dashboard > select your space > click Space Settings > click the API Details tab > click the Create a New Key button under Private Token in the Data Ingest API section. The key will only be displayed once, so be sure to save it in a secure location
Starting an index
To start an index, use the POST
https://ingest.api.nacelle.com/v1/index
endpoint. In an entryTypes
JSON Schema, define the type of index you want to create. This is where you evaluate the type of data that you are going to include in the indices by looking at what you plan to do with that data. Valid values include:
COLLECTION
: Specifies a grouping or category of related items or products.COLLECTION_CONTENT
: Content associated with a specific collection, such as descriptions or images.
CONTENT
: Content and copy ingested from a CMS.PRICE_BOOK
: Defines a set of prices for products, typically used for different pricing strategies or customer segments.PRICING
: Pricing information associated with products or variants.PRODUCT
: Individual items available for sale.PRODUCT_CONTENT
: Content specific to a particular product, including details, descriptions, and media.
PRODUCT_VARIANT
: Variations of a product, such as different sizes or colors.PRODUCT_VARIANT_CONTENT
: The content specific to a product variant, such as variant-specific details or images.
CUSTOM_SCHEMA
: Allows for the ingestion of data structured according to a custom-defined schema, enabling flexibility in data organization and representation.
Example
Uses the schema from Start an Index
.
Request
curl --request POST \
--url https://ingest.api.nacelle.com/v1/index \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--header 'x-nacelle-ingest-token: {your ingest token}' \
--header 'x-nacelle-source-id: {your source ID}' \
--header 'x-nacelle-space-id: {your space ID}' \
--data '
{
"entryTypes": [
"PRODUCT",
"COLLECTION",
"PRODUCT_VARIANT"
]
}
Response
{
"message": "Index Created",
"indexId": "{your new index ID}"
}
Finishing an Index
You can only have one active index at a time. To make an index your active index, use the POST Finish an Index
endpoint.
Example
Uses the schema from Finish an Index
.
Request
curl --request POST \
--url https://ingest.api.nacelle.com/v1/index/b9367bc5-ac9c-4409-9016-fb23f8819314/finish \
--header 'accept: application/json' \
--header 'x-nacelle-ingest-token: {your ingest token}' \
--header 'x-nacelle-source-id: {your source ID}' \
--header 'x-nacelle-space-id: {your space ID}' \
Response
{
"message": "Index Finished"
}
Adding content, collections, products, and pricing data
Nacelle's data ingestion system uses a hierarchical structure to manage content, collections, products, and pricing data. This system ensures that your commerce data is organized and easily accessible. Now that you've created an index, you can begin adding your content, collections, products and pricing data to it. The logic is that you use separate endpoints to add top-level entries such as content, collections, products, and pricing data to the index. Most of these top-level entries (except content) can then have additional content added to them through separate endpoints. You can update the content of these entries at any time to keep your data current and relevant.
- Product - Endpoint to add product entries
- Product Content - Endpoint to add product content
- Variant - Endpoint to add product variants. For example, sizes, colors or types.
- Variant Content - Endpoint to add product variant content.
- Collection - Endpoint to add a collection of products. For example, all shoes, or all bags.
- Collection Content - Endpoint to add content to a product collection of products.
- Content - Endpoint to add CMS content to your index.
Example
Uses the schema from Add a Product
, and Add Product Content
.
Request
curl --request PUT \
--url https://ingest.api.nacelle.com/v1/product \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--header 'x-nacelle-ingest-token: {your ingest token}' \
--header 'x-nacelle-source-id: {your source ID}' \
--header 'x-nacelle-space-id: {your space ID}' \
--data '
{
"indexId": "e8a52947-42f8-439d-bf02-98098d32014f",
"entries": [
{
"taxes": {
"taxProvider": "Avalara"
},
"sourceEntryId": "gid://shopify/Product/0000000000000",
"availableForSale": false
}
]
}
Response
{"message":"PRODUCT UPDATE: success"}
curl --request PUT \
--url https://ingest.api.nacelle.com/v1/product-content \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--header 'x-nacelle-ingest-token: {your ingest token}' \
--header 'x-nacelle-source-id: {your source ID}' \
--header 'x-nacelle-space-id: {your space ID}' \
--data '@product.json'
Example product.json
{
"entries": [
{
"featuredMedia": {
"src": "https://images.salsify.com/image/upload/s--8qOuPa0f--/3bff7c48b0ade02ff35783f29ac08352de8b44cd.jpg",
"thumbnailSrc": "https://images.salsify.com/image/upload/s--8qOuPa0f--/3bff7c48b0ade02ff35783f29ac08352de8b44cd.jpg",
"type": "IMAGE"
},
"sourceProductId": "gid://shopify/Product/0000000000000",
"sourceEntryId": "3858f62230ac3c915f300c664312c63f",
"title": "Wayfarer Messenger Bag",
"description": "<p>Perfect for class, work or the gym, the Wayfarer Messenger Bag is packed with pockets. The dual-buckle flap closure reveals an organizational panel, and the roomy main compartment has spaces for your laptop and a change of clothes. An adjustable shoulder strap and easy-grip handle promise easy carrying.</p>\\n<ul>\\n<li>Multiple internal zip pockets.</li>\\n<li>Made of durable nylon.</li>\\n</ul>",
"media": [
{
"src": "https://images.salsify.com/image/upload/s--8qOuPa0f--/3bff7c48b0ade02ff35783f29ac08352de8b44cd.jpg",
"thumbnailSrc": "https://images.salsify.com/image/upload/s--8qOuPa0f--/3bff7c48b0ade02ff35783f29ac08352de8b44cd.jpg",
"type": "IMAGE"
}
],
"metafields": [
{
"key": "Brand",
"value": "Luma",
"namespace": "filter"
},
{
"key": "vipPriceFactor",
"value": "0.9",
"namespace": "segmentPrice"
}
]
}
]
}
Response
{
"message": "PRODUCT_CONTENT UPDATE: success"
}
Support
If you get stuck, send us an email or use the Intercom widget on the bottom right of your Nacelle Dashboard.
What's next
- To understand what happens to your data once ingested, check out thecanonical commerce data model
- To begin building your storefront, see the Order Management introduction
- For account settings, see Account, access control, and dashboard setup
Updated 3 months ago