{"id":8586,"date":"2025-03-18T15:46:25","date_gmt":"2025-03-18T15:46:25","guid":{"rendered":"https:\/\/www.process.st\/help\/?post_type=ht_kb&#038;p=8586"},"modified":"2025-03-18T16:51:56","modified_gmt":"2025-03-18T16:51:56","slug":"data-management-data-set-automations","status":"publish","type":"ht_kb","link":"https:\/\/www.process.st\/help\/docs\/data-management-data-set-automations\/","title":{"rendered":"Optimize Data Management with Data Set Automations"},"content":{"rendered":"<h3><a href=\"#guide\">Skip to the guide \u2192<\/a><\/h3>\n<h2><strong>The Challenges of Managing Data in Process Street<\/strong><\/h2>\n<p>Process Street&#8217;s Data Sets help centralize and reuse data across workflows. However, unlike dedicated databases, Data Sets have limitations that can slow teams down:<\/p>\n<ol>\n<li><strong>Manual updates are time-consuming:<\/strong> Keeping Data Sets current is tedious, especially for dynamic data like client lists, product catalogs, or team rosters.<\/li>\n<li><strong>Limited data validation:<\/strong> Data Sets don\u2019t include advanced validation features, making it easier for errors like duplicates or inconsistent data to slip through.<\/li>\n<li><strong>No advanced queries:<\/strong> Data Sets offer basic filtering but lack the robust query tools found in most databases, which are often essential for managing and analyzing complex data structures.<\/li>\n<\/ol>\n<h3><strong>Most Businesses Already Store Their Data Elsewhere<\/strong><\/h3>\n<p>Most critical business data already lives in CRMs, ERPs, or databases. Managing it separately in Process Street creates duplication and inconsistencies. Integrating these sources eliminates manual syncing, ensuring workflows stay accurate and up to date.<\/p>\n<h3><strong>The Solution: Dynamic Updates<\/strong><\/h3>\n<p>By connecting Process Street to an external data source and automating updates, you can:<\/p>\n<ol>\n<li><strong data-start=\"148\" data-end=\"172\">Keep Data Up to Date: <\/strong>Dropdowns automatically reflect the latest information, so you don\u2019t have to update them manually.<\/li>\n<li><strong data-start=\"278\" data-end=\"307\">Reduce Errors &amp; Save Time: <\/strong>Dynamic dropdowns ensure accurate, pre-validated selections, making workflows more efficient.<\/li>\n<li><strong data-start=\"408\" data-end=\"452\">Use Your Existing Tools More Effectively: <\/strong>Instead of manually copying data, integrate updates directly from your CRM or other systems.<\/li>\n<\/ol>\n<h3><strong>Scale Without the Manual Effort<\/strong><\/h3>\n<p>Managing large Data Sets manually becomes more challenging as your business grows. Automating updates keeps your workflows accurate while freeing your team to focus on more important tasks. With dynamic updates, dropdowns always reflect the latest data. No extra work is required.<\/p>\n<p data-start=\"355\" data-end=\"580\">By linking Process Street with external data sources, you combine easy-to-use workflows with powerful data management from your existing systems.<\/p>\n<p><!-- notionvc: 1122cae8-acba-4110-a612-7934a181b434 --><\/p>\n<h2 id=\"guide\"><strong>How to Dynamically Update a Data Set<\/strong><\/h2>\n<p data-start=\"52\" data-end=\"245\">This guide is for users who don\u2019t use or have access to Zapier. I\u2019ll be using Power Automate to send HTTP requests to Process Street\u2019s API, but any tool that can send HTTP requests will work.<\/p>\n<h3><strong>Efficient Data Set Syncing: A Smarter Approach<\/strong><\/h3>\n<p>Below is my typical sync flow for creating, modifying, and deleting records in a Data Set. Every action first goes through the System of Record before updating the Data Set. While updating the Data Set directly from a Process Street Workflow Run might seem more efficient, this approach actually reduces the overall automation work required.<br \/>\nInstead of creating multiple automations, I build a single automation that monitors the System of Record for any new or modified records. When a change is detected, the automation locates the corresponding Data Set record and updates it accordingly. This streamlined method simplifies the process and ensures the Data Set stays in sync with your source of truth.<br \/>\n<!-- notionvc: 7193a867-cac8-4e37-ac52-5cbb8b55616f --><br \/>\n<img decoding=\"async\" class=\"alignnone wp-image-94158 \" src=\"https:\/\/www.process.st\/wp-content\/uploads\/2025\/03\/dynamically-update-datasets.webp\" alt=\"dynamically update datasets\" width=\"100%\" \/><\/p>\n<h2><strong>Step 1: Set Up Your External Data Source &amp; Data Set<\/strong><\/h2>\n<p>Start by defining how you want your data to be structured and managed. Consistency matters\u2014without it, your external source and Process Street Data Set will fall out of sync.<br \/>\nExcel and Google Sheets work, but structured databases or tables are better. They enforce formatting and prevent accidental changes.<\/p>\n<p data-start=\"297\" data-end=\"391\">When setting up your Data Set, don\u2019t sync everything. Only pull in what your workflow needs:<\/p>\n<ul>\n<li data-start=\"393\" data-end=\"488\"><strong data-start=\"395\" data-end=\"413\">Dropdown Field<\/strong>: The value displayed in the dropdown (e.g., client names, product SKUs).<\/li>\n<li data-start=\"489\" data-end=\"608\"><strong data-start=\"491\" data-end=\"501\">Row ID<\/strong>: The unique identifier from the source system (e.g., primary key, unique ID). This keeps records linked.<\/li>\n<li data-start=\"609\" data-end=\"716\"><strong data-start=\"611\" data-end=\"628\">Filter Fields<\/strong>: Extra fields to create filtered views (e.g., active vs. inactive items, categories).<\/li>\n<\/ul>\n<p>For easier maintenance, keep column names similar between the external source and the Data Set. They don\u2019t have to match exactly, but close alignment makes mapping and troubleshooting smoother.<\/p>\n<h2><strong>Step 2: Connect the External Source to the Data Set<\/strong><\/h2>\n<p>To keep your Process Street Data Set up-to-date, you can use webhooks to automate creating and updating records from your external source. Webhooks can create and update records, but they won\u2019t delete them. For deletions, the Process Street API works, but Zapier makes it easier.<br \/>\nHere\u2019s how to connect your external source to the Data Set:<\/p>\n<h3><strong>1. Set Up a Webhook in the Process Street Data Set<\/strong><\/h3>\n<p><a href=\"https:\/\/www.process.st\/help\/docs\/data-set-automations\/#creating-a-data-set-record-with-a-webhook-trigger\">Create a webhook<\/a> in the automation settings of your Process Street Data Set. This webhook will receive data from the external source whenever a change occurs. The only configuration you\u2019ll need to do will be when you send your first test data from Power Automate. Go ahead and copy the webhook\u2019s URL and keep this window open so you can test the trigger later.<br \/>\n<!-- notionvc: 852d5d34-0f4e-4469-b159-c28d5d1825cc --><br \/>\n<img decoding=\"async\" class=\"alignnone wp-image-94159\" src=\"https:\/\/www.process.st\/wp-content\/uploads\/2025\/03\/process-street-dynamic-workflows-datasets-drop-downs.webp\" alt=\"\" width=\"100%\" \/><\/p>\n<h3><strong>2. Create Your Syncing Automation<\/strong><\/h3>\n<ol>\n<li>Open Power Automate and create a new flow.<\/li>\n<li>Select a trigger that aligns with your external source (e.g., \u201cWhen a row is added or modified\u201d for databases or \u201cWhen a file is updated\u201d for Excel\/Google Sheets).<\/li>\n<li>Add an <strong>HTTP action<\/strong> to your flow and configure it as follows:\n<ol>\n<li><strong>Method:<\/strong> <code>POST<\/code><\/li>\n<li><strong>URI:<\/strong> Paste the Process Street webhook URL.<\/li>\n<li><strong>Headers:<\/strong> Set Content-Type to <code>application\/json<\/code>.<\/li>\n<li><strong>Body:<\/strong> Format the payload with the data you want to send. For example:<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<p><!-- notionvc: 31ef1aec-06f3-4daa-b63c-b9d6c9cebcc2 --><\/p>\n<p style=\"padding-left: 80px;\">{<br \/>\n&#8220;row_id&#8221;: &#8220;12345&#8221;,<br \/>\n&#8220;title&#8221;: &#8220;Example&#8221;,<br \/>\n&#8220;status_field&#8221;: &#8220;Active&#8221;<br \/>\n}<\/p>\n<p style=\"padding-left: 80px;\"><em>Use dynamic content in Power Automate to replace these placeholders with data from your external source.<!-- notionvc: b7024676-a1ec-4fde-bed3-4b910789d780 --><\/em><\/p>\n<p><img decoding=\"async\" class=\"alignnone wp-image-94160\" src=\"https:\/\/www.process.st\/wp-content\/uploads\/2025\/03\/dynamically-update-datasets-process-street.webp\" alt=\"\" width=\"100%\" \/><\/p>\n<h3><strong>3. Test the Power Automate Flow &amp; Configure the Webhook<\/strong><\/h3>\n<ol>\n<li>Run a test in Power Automate to confirm the data is hitting the Process Street webhook correctly.<\/li>\n<li>Switch back to Process Street and click <code>Test Trigger<\/code>. This will bring in the data just sent by your flow test.<\/li>\n<li>Configure the webhook. If everything&#8217;s set up right, the mapping will be a direct 1-to-1 from the JSON payload.<\/li>\n<\/ol>\n<p>Process Street\u2019s UI makes this part simple. If the data structure matches, it should work without issues.<br \/>\n<!-- notionvc: d3c128c2-61ad-4819-80cc-b7ebf6a3f715 --><br \/>\n<img decoding=\"async\" class=\"alignnone wp-image-94161\" src=\"https:\/\/www.process.st\/wp-content\/uploads\/2025\/03\/updating-data-sets-process-street-drop-downs.webp\" alt=\"updating data sets process street\" width=\"100%\" \/><\/p>\n<h2><strong>Step 3: Manage Deletions Separately<\/strong><\/h2>\n<p>Process Street webhooks won&#8217;t delete records, so you&#8217;ll need another approach to remove outdated entries:<\/p>\n<ul>\n<li data-start=\"129\" data-end=\"274\"><strong data-start=\"131\" data-end=\"153\">Process Street API<\/strong>: Programmatically delete records. This takes some technical know-how or an integration tool that can handle API calls.<\/li>\n<li data-start=\"275\" data-end=\"357\"><strong data-start=\"277\" data-end=\"296\">Manual Cleanups<\/strong>: Periodically review and remove outdated entries yourself.<\/li>\n<\/ul>\n<p>I&#8217;m putting together a guide on setting up and automating row deletions with the Process Street API. It&#8217;ll walk you through using my filter query automation to pinpoint records for removal, making cleanup quick and painless. Stay tuned.<\/p>\n<h2><strong>Keeping Your Data Set Accurate &amp; Effortless<\/strong><\/h2>\n<p data-start=\"26\" data-end=\"168\">Webhooks keep your Process Street Data Set accurate\u2014no manual updates, no errors. They handle updates, but deletions need Zapier or the API. With this setup, dropdowns stay current, workflows stay clean, and your team works with reliable data.<\/p>\n<h3 data-start=\"26\" data-end=\"168\"><strong>About the Author<\/strong><\/h3>\n<p><a href=\"https:\/\/www.linkedin.com\/in\/desilvakai\/\">Kai deSilva<\/a> is The Creative Guy at <a href=\"https:\/\/www.linkedin.com\/company\/improvizations\/\">Improvizations,<\/a> a UKG-focused advisory and implementation firm expanding into AI-driven process consulting and optimization. With a background in graphic design and a passion for automation, Kai helps businesses\u2014ranging from manufacturing and financial services to education and marketing\u2014streamline workflows and enhance efficiency with AI. When he&#8217;s not optimizing processes, you&#8217;ll find him lifting weights, riding motorcycles, or exploring the intersection of creativity and tech.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Skip to the guide \u2192 The Challenges of Managing Data in Process Street Process Street&#8217;s Data Sets help centralize and reuse data across workflows. However, unlike dedicated databases, Data Sets have limitations that can slow teams down: Manual updates are time-consuming: Keeping Data Sets current is tedious, especially for dynamic data like client lists, product [&hellip;]<\/p>\n","protected":false},"author":11,"template":"","meta":{"_acf_changed":false,"available_plan":"All plans","available_product":"Data Sets, Integrations","footnotes":""},"ht_kb_category":[194,472],"ht_kb_tag":[480],"class_list":["post-8586","ht_kb","type-ht_kb","status-publish","hentry","ht_kb_category-integrations","ht_kb_category-data-sets","ht_kb_tag-data-sets"],"acf":[],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.process.st\/help\/wp-json\/wp\/v2\/ht_kb\/8586","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.process.st\/help\/wp-json\/wp\/v2\/ht_kb"}],"about":[{"href":"https:\/\/www.process.st\/help\/wp-json\/wp\/v2\/types\/ht_kb"}],"author":[{"embeddable":true,"href":"https:\/\/www.process.st\/help\/wp-json\/wp\/v2\/users\/11"}],"version-history":[{"count":3,"href":"https:\/\/www.process.st\/help\/wp-json\/wp\/v2\/ht_kb\/8586\/revisions"}],"predecessor-version":[{"id":8589,"href":"https:\/\/www.process.st\/help\/wp-json\/wp\/v2\/ht_kb\/8586\/revisions\/8589"}],"wp:attachment":[{"href":"https:\/\/www.process.st\/help\/wp-json\/wp\/v2\/media?parent=8586"}],"wp:term":[{"taxonomy":"ht_kb_category","embeddable":true,"href":"https:\/\/www.process.st\/help\/wp-json\/wp\/v2\/ht_kb_category?post=8586"},{"taxonomy":"ht_kb_tag","embeddable":true,"href":"https:\/\/www.process.st\/help\/wp-json\/wp\/v2\/ht_kb_tag?post=8586"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}