Community
Questions Library
Docs
Blog
Events
Swag
Github
Slack
JupiterOne
Discussions
Release Notes
Contact Us
How to configure and run a managed integration locally - AskJ1 Community
<main> <article class="userContent"> <p>There are a number of reasons why a JupiterOne managed integration should be run locally. Some examples include the tool is running within a VPC, firewall rules don’t allow external access, or the tool is running on-prem. This article outlines the steps for running an integration locally. These steps include extracting the data, viewing them as a graph, and loading the data back into JupiterOne’s graph.</p> <p>You can find additional developer documentation here in our JupiterOne SDK. This documentation, along with some of the commands we’ll review and pre-requisites in the JupiterOne platform will be needed when creating your own custom integrations.</p> <h2 data-id="set-up-in-jupiterone">Set up in JupiterOne</h2> <p>Before you can send the integration results into JupiterOne, you will need your <strong>JupiterOne Account ID</strong> and you will need to create a Mock Integration Instance so that you can capture the <strong>Integration Instance ID</strong> and create an <strong>Integration API Key</strong>.</p> <ol><li>Your JupiterOne Account ID</li> </ol><p>Find this ID by navigating to the account management page or running this query: <br></p> <p><code class="code codeInline" spellcheck="false" tabindex="0">FIND jupiterone_account as a return a._accountId</code></p> <ol start="2"><li>An Integration Instance ID</li> <li>An Integration API Key</li> </ol><p>For 2 and 3, follow these steps:</p> <h3 data-id="create-a-template-integration-instance-using-dummy-data">Create a template integration instance using dummy data:</h3> <ol><li>Navigate to the integrations page and select the integration you are going to configure and click the add integration instance button (we’ll use Jira as an example throughout this article).</li> <li>Fill in the all required fields (see the screenshot for an example)</li> </ol><ul><li>Fill in the Account Name field with a value that includes the word custom, manual, or local. You especially want a descriptive name if you have multiple integration instances configured for the same tool.</li> <li>Set the Polling Interval to be Disabled so that the JupiterOne platform does not try to run the integration at a specific interval.</li> <li>Fill in the rest of the fields with dummy data. The create configuration button is not clickable until all required fields have been filled in or selected.</li> </ul><p><img src="https://us.v-cdn.net/6035534/uploads/D96W58T8UYCG/local-managed-integration-create.png" alt="" class="embedImage-img importedEmbed-img"></img></p> <ol start="3"><li>After you create the configuration, you should see an ID field appear. This is the Integration Instance ID. You’ll need this value later.</li> </ol><p><img src="https://us.v-cdn.net/6035534/uploads/NDSFTF2AHH6E/local-managed-integration-integration-id.png" alt="" class="embedImage-img importedEmbed-img"></img></p> <ol start="4"><li>You will also see towards the bottom of the page a section to create an integration API key. Click the create button.</li> </ol><p><img src="https://us.v-cdn.net/6035534/uploads/4MIZ93S7VQZ7/local-managed-integration-api-key-create.png" alt="" class="embedImage-img importedEmbed-img"></img></p> <p>You can now click to reveal the Integration API Key. You'll also need this value later.</p> <p><img src="https://us.v-cdn.net/6035534/uploads/I5GXP8EV6ELQ/local-managed-integration-api-key-view.png" alt="" class="embedImage-img importedEmbed-img"></img></p> <h2 data-id="cloning-and-running-the-integration">Cloning and Running the Integration</h2> <p>Next you'll clone the repository of the integration you are trying to run locally. Integration repositories are public and can be found under the <a rel="nofollow" href="https://github.com/JupiterOne">JupiterOne Organization</a>. When you search for an integration repository, you'll notice each follows a naming convention of <strong>graph-*</strong>. We’ll continue with our Jira example by navigating to the <a rel="nofollow" href="https://github.com/JupiterOne/graph-jira">Jira Repository</a>.</p> <p>Copy the link under Code so we can clone the repository.</p> <p><img src="https://us.v-cdn.net/6035534/uploads/F1Q5ZS0F3JDI/local-managed-integration-graph-jira.png" alt="" class="embedImage-img importedEmbed-img"></img></p> <p>We'll describe two methods for installing dependencies and running the integrtion: running the job locally using a Mac terminal or using a provided Dockerfile to run the job from within a Docker container.</p> <h3 data-id="using-the-mac-terminal">Using the Mac Terminal</h3> <ol><li>Make sure all prerequisites are installed.</li> </ol><ul><li>Install <a rel="nofollow" href="https://brew.sh/">Homebrew</a> (this will take a moment)</li> <li>Install <a rel="nofollow" href="https://nodejs.org/en/">Node.js</a> using the <a rel="nofollow" href="https://nodejs.org/en/download/">installer</a> or a version manager such as <a rel="nofollow" href="https://github.com/nvm-sh/nvm">nvm</a> or <a rel="nofollow" href="https://github.com/Schniz/fnm">fnm</a></li> <li>Install yarn or npm: <code class="code codeInline" spellcheck="false" tabindex="0">brew install yarn</code> or <code class="code codeInline" spellcheck="false" tabindex="0">brew install npm</code></li> </ul><ol start="2"><li>Next, open a terminal in the location where you'd like the repo to live. Type in <code class="code codeInline" spellcheck="false" tabindex="0">git clone</code> followed by the url copied from GitHub.</li> </ol><p><code class="code codeInline" spellcheck="false" tabindex="0">git clone https://github.com/JupiterOne/graph-jira.git</code></p> <p>Remember which directory you clone the repo to. You’ll want to perform the next steps in the root of the cloned repository.</p> <ol start="3"><li>Install dependencies (you can skim through the package.json file to see which dependencies are being installed)</li> </ol><p><code class="code codeInline" spellcheck="false" tabindex="0">yarn install</code> or <code class="code codeInline" spellcheck="false" tabindex="0">npm install</code></p> <ol start="4"><li>Copy the contents of the environment example file to a new environment file which will be used when running the integration.</li> </ol><p><code class="code codeInline" spellcheck="false" tabindex="0">cp .env.example .env</code></p> <ol start="5"><li>Open the environment file to edit the values of the listed parameters (and add additional environment variables as needed).</li> </ol><p><code class="code codeInline" spellcheck="false" tabindex="0">vi .env</code></p> <p>If not listed, these parameters are required across all integrations:</p> <ul><li>JUPITERONE_API_KEY</li> <li>JUPITERONE_ACCOUNT</li> <li>INTEGRATION_INSTANCE_ID</li> </ul><p>There are also some integration specific values that are listed in the .env.example file. For example, Jira requires:</p> <ul><li>JIRA_HOST</li> <li>JIRA_USERNAME</li> <li>JIRA_PASSWORD</li> <li>PROJECTS</li> <li>JIRA_API_VERSION</li> <li>BULK_INGEST_ISSUES</li> </ul><p><em>You may notice a few examples where the .env.example file is outdated and does not list out all of the required parameters. There may also be times when it is not clear what format the value of a parameter should be. In these situations, please skim through the config.ts file in the src folder of a graph-* integration repo. Here is an <a rel="nofollow" href="https://github.com/JupiterOne/graph-jira/blob/main/src/config.ts">example config.ts</a> file from the graph-jira repo. Parameters use the UPPERCASE_EXAMPLE_SYNTAX.</em></p> <p><strong>Now that everything is set up, you are ready to trigger the integration job.</strong></p> <ol><li>Collect data from the source tool</li> </ol><p><code class="code codeInline" spellcheck="false" tabindex="0">yarn start or yarn j1-integration collect</code></p> <ol start="2"><li>Push the data into JupiterOne</li> </ol><p><code class="code codeInline" spellcheck="false" tabindex="0">yarn j1-integration sync</code></p> <ol start="3"><li>(Optional) Visualize the collected data</li> </ol><p><code class="code codeInline" spellcheck="false" tabindex="0">yarn graph or yarn j1-integration visualize</code></p> <ol start="4"><li>For most repositories, you can skip steps 1 and 2 and instead just call the run command which includes both the start and sync steps</li> </ol><p><code class="code codeInline" spellcheck="false" tabindex="0">yarn j1-integration run</code></p> <p><strong>Finally, navigate to the integration instance in JupiterOne to view the data that has been pushed into the platform.</strong></p> <h3 data-id="using-a-dockerfile">Using a Dockerfile</h3> <p>Docker is just one example of how you can automate the environment setup and execution of the previous steps. <a rel="nofollow" href="https://github.com/ajahoda/j1-graph-docker">This Docker repository</a> walks you through the process of copying over its contents into your cloned integration repository, building a Docker image, and running a Docker container to collect information from the integration and sync it with the JupiterOne platform.</p> <h2 data-id="resources">Resources</h2> <p>The <a rel="nofollow" href="https://github.com/JupiterOne/sdk/blob/main/docs/integrations/development.md">JupiterOne SDK</a> describes how graph-* integration repositories are configured and provides information on how to build, test, and execute. The section <a rel="nofollow" href="https://github.com/JupiterOne/sdk/blob/main/docs/integrations/development.md#supported-commands">supported commands</a> is especially useful because it describes what options you can pass to execute different scripts.</p> </article> </main>
.png)