Diffbot Zapier integrationAutomate Diffbot with Zapier.
The Diffbot Zapier integration gives you 1 trigger, 0 actions and 0 searches. One event, then: a Crawlbot crawl has finished. Small, yes. But it fires instantly on every plan, and it is the cue most crawl jobs are missing.
Verified Trustpilot reviews · AI, automation & growth agency
Your crawls finish while nobody is watching
Connecting Diffbot to Zapier means one thing: when Crawlbot, the Diffbot spider that walks a whole site, finishes a crawl, Zapier hears about it and runs whatever steps you placed after it. A Zap is that automation, built from one starting event and the steps that follow it. Nothing else in Diffbot can start one.
Three results come up again and again, all of them built on New Crawl Notification. First, a ping in a team channel, so the analyst who launched a crawl of a competitor's catalog stops refreshing the dashboard. Second, a new row in a shared tracking sheet, one per finished crawl, which turns into a cheap log of what ran and when. Third, an email to the account manager, sent only for the crawls tied to one client.
Now the honest part. Zapier does not start a crawl, does not extract a page and does not fetch crawl results for you: the app offers no action and no search at all. The event also covers crawls only. The Diffbot description mentions bulk extraction jobs too, but no Zapier event exists for them. So the crawl itself stays a Diffbot job, and Zapier handles the follow-up.
If you are weighing tools, the same app has a Diffbot Make integration page, and the Make vs Zapier comparison sets the two side by side on general criteria.
Zapier vocabulary in one minute
Eight words, and the rest of this page reads easily. Each one shows up in the editor.
- Zap
- The automation you build on Zapier: one starting event, then one or more steps that run each time that event happens.
- Trigger
- The event in an app that starts the Zap. Here it is a finished Crawlbot crawl; once the Zap is on, Zapier waits for it.
- Action
- What the Zap does once it has started: the step that writes a row, posts a message or sends an email somewhere else.
- Task
- What Zapier bills: one action that completes successfully. The allowance on your plan is a count of those, nothing else.
- Polling vs instant
- A polling trigger is checked on a timer. An instant trigger is reported by the app itself the moment it happens. Diffbot's is instant.
- Filter
- A step that lets the Zap carry on only when what arrived meets your condition, and quietly stops it the rest of the time.
- Connection
- Your Diffbot account linked to Zapier once, then reused by every Zap and every step that needs Diffbot.
- Multi-step Zap
- Any Zap beyond one trigger and one action. A filter or a second action makes it one, and it then needs a paid plan.
What a crawl alert really costs
The trigger never costs a task. Only the actions that work are counted, one task each.
- Trigger0 tasks
- Each successful action1 task
- Free plan100 tasks a month, two-step Zaps
- Premium appNo
Diffbot is not a premium app, so it runs on the free plan: 100 tasks a month, as many Zaps as you like, but two steps per Zap. That covers the plain case: a finished crawl, then one message. A filter makes it multi-step, which means a paid plan, and so does a second action. A failed action is not counted. Since the trigger is instant, the interval grid below never slows it down, even on Free. Keep in mind that every finished crawl on the account runs the Zap, so a busy account eats tasks faster than you think.
Connect Diffbot to Zapier in three steps
- 01
Open the Apps page
In your Zapier account, go to Apps and click + Add connection. A connection is your Diffbot account linked once, then reused by every Zap you build on it.
- 02
Pick Diffbot
Type Diffbot in the search box of the dialog, select it, then click Add connection. You need an existing Diffbot account for the rest to work, since Zapier does not create one.
- 03
Sign in and allow access
A tab opens on Diffbot. Sign in there, then grant Zapier access to the account. If the crawls belong to a teammate, make sure your login can actually see them.
Your first Zap: a Slack ping for one client's crawls
GoalWhen a crawl run for a given client finishes, the SEO team's Slack channel gets a message.
- 01
Build: pick the trigger
Create a Zap, choose Diffbot as the starting app and New Crawl Notification as the event. Select the connection you added earlier.
- 02
Test: fetch real crawls
Run the test. Zapier brings back the three most recent items from your account, so let at least one crawl finish beforehand and pick it as your sample.
- 03
Add a filter
Click the plus icon, choose Filter, then set the field to watch, the rule and the value. Use a text rule on a text value, otherwise the filter never passes.
- 04
Add the Slack action
Add Slack with Send Channel Message, pick the channel, and drop values from the Diffbot sample into the message so the team knows which crawl just ended.
- 05
Publish
Test the Slack step once and read what came back. When it looks right, click Publish: the Zap is on and waits for the next finished crawl.
The Diffbot trigger
1 event starts a Zap when something happens in Diffbot, and one of them is instant: the app calls Zapier straight away, whatever your plan.
New Crawl Notification
Instant triggerIn Zapier“Your webhook URL will be notified when your Crawlbot crawl finishes.”
Starts your Zap each time a Crawlbot crawl reaches its end. It is the only Diffbot event Zapier offers, and it tells you a job is done rather than handing you the pages the crawl collected.
When the Diffbot Zap breaks
No Zapier help article is attached to Diffbot, so the failures below are the general Zapier ones that matter for this trigger.
Most problems come from the plan or the filter, rarely from Diffbot itself. Four cases cover what you are likely to hit. For anything else, the Zapier troubleshooting guide walks through a broken Zap step by step.
Old crawls never show up
A Zap only reacts to crawls that end after it is switched on. Jobs finished before that are not picked up, and logging them is a separate task.A burst of crawls gets held back
If many crawls end at once, say after you launched a big batch, Zapier's flood protection can hold the trigger back instead of firing for each one.The filter stops working on a free plan
A filter turns the Zap into a multi-step Zap. The free plan only builds two steps, so that Zap needs a paid plan to keep running.The filter never lets anything through
Filter rules are typed. A number rule on a text value, or the reverse, fails every time. Check the rule type before blaming Diffbot.
Need help automating Diffbot with Zapier?
A person reads every message.
