Getting Started With Analytics 2.0 (Beta)

Table of Contents

Overview

 In Beta
Existing Analytics customers have the opportunity to participate in the Open Beta for Analytics 2.0. Please reach out to your Customer Success Manager to learn more.

Analytics 2.0 provides a powerful, user-friendly solution for leveraging your project data, making informed decisions, and gaining deeper insights into your overall project performance and outcomes. 

Key features include:

  • Data refreshed every hour that you can pull at any time. Plus, data is monitored for accuracy and completeness.
  • All Procore data can be pulled out, including custom fields, workflows, change history and user activity.*
  • New data platform with significantly increased scalability and data availability.
  • Pre-built Power BI reporting.

*All datasets won't be available at the start of open beta. Full feature parity with Analytics 1.0 coming soon.

 

 

Considerations

  • You can continue using your current version of Procore Analytics, even if you opt into the Open Beta for Analytics 2.0.
     
  • If you're not secbng the callout to join the open beta, another Company Admin in your company may have already declined the beta agreement.
     
  • To re-activate the invitation, contact your Customer Success Manager for reactivation.
     
  • If you're having trouble connecting to Procore Analytics 2.0, contact your Procore Customer Success Manager.

Common Questions

How frequent is the data refresh?

Data refreshes throughout the day on an hourly basis. 

How do I connect to Procore Analytics 2.0?

Analytics 2.0 can be accessed in several ways. The most common way would be to connect in Power BI. See Getting Started With Procore Analytics 2.0. 

Will I lose access to existing reporting?

Procore Analytics 1.0 will be available until 2025. We aim to give users plenty of time to switch over to Analytics 2.0.

Will customers on Data Extract have to migrate to Procore Analytics 2.0?

Yes, customers on Data Extract will need to move over to Procore Analytics 2.0, now called Cloud Connector (powered by Delta Share).  Procore Analytics 2.0 will also include PowerBI reports as a feature that customers are not required to use, but they will be available for all customers with Procore Analytics 2.0.

Do I still need Power BI licenses to access Procore Analytics2.0?

Power BI licenses will still be required to share Power BI reports. 

How do I update the custom reports that were developed in-house?

Procore Analytics 2.0 now uses Databricks Delta Sharing. Any existing connections to Analytics 1.0 (Azure SQL) must be changed. Several report tables and columns have been renamed. The basic changes you will need to make include updating reports to use the new Delta Sharing connector, updating table and column names, and updating visuals that may have broken after you've made the changes mentioned above. Depending on how your report was built, there may be additional considerations. See Getting Started With Procore Analytics 2.0. 

How will the process for copying data to my data warehouse be affected by 2.0?

Procore Analytics 2.0 now uses Databricks Delta Sharing for data access. You can find out more about delta sharing here.    

 

For any further questions or concerns regarding Procore Analytics 2.0, please reach out to the Analytics 2.0 Beta Community Group. 

Permissions

Learn which user permissions are required to take the described actions in this tool.

 

Permissions

 |  |  The action is available on Procore's Web, iOS, and/or Android application. Click to view the article.

icon-mindtouch-table-check.png Users can take the action with this permission level.

icon-mindtouch-table-check.png Users can take this action with this permission level AND one or more additional requirements, like granular permissions.

Action None Read Only Standard Admin Notes

Generate A Data Token

 Web

       check-green2.png

Anyone with 'Admin' level access to Analytics can grant additional users access to the Analytics tool. 

 

Get Access to Your Procore Data

To start accessing your Procore data, you must generate an access token. The access token is a string of digits you will enter in your BI system to access data. Typically, users who need access tokens are data engineers or Power BI developers. 

 

Considerations

  • You must have the Procore Analytics tool enabled. 
  • To enable Analytics 2.0, a Company Admin must first accept the Open Beta agreement.
  • By default, all Company Admins have 'Admin' level access to Analytics in the Directory. 
  • Anyone with 'Admin' level access to Analytics can grant additional users access to the Analytics tool. 
  • Users must have 'Admin' level access to the Analytics tool to generate a data token. 

Steps

Accept Beta Agreement and Activate

Once invited to the open beta, Company Admins will see a popup banner inviting you to accept the beta agreement the next time you log in to Procore. If you do not see the popup, another Company Admin may have already declined the beta agreement. Please reach out to your Customer Success Manager to get reactivated.

Grant Data End-Users Admin Level Access to Analytics 2.0

Once any Company Admin accepts the beta agreement, all Company Admins will see Procore Analytics in the tool dropdown menu within Procore. To add Analytics Admins (e.g. someone in IT, or a data analyst) to assist with your next steps, navigate to the user's Company level Directory card and give them Admin-level access to the Analytics tool.

Connect to Your Data 

Anyone with Admin-level access to the Analytics tool will be able to generate data access tokens through the Analytics tool. Data end users should generate data tokens and will then need to take the additional steps in the next section of this guide to connect the data to a data warehouse or BI solution. To begin connecting your data, follow the steps below:

  1. Log in to Procore. 
  2. Click the Account & Profile icon in the navigation bar.
  3. Click My Profile Settings. 
  4. Click the Procore Analytics Tokens tab. 
  5. Next you will choose your connection with Procore Analytics.
  6. If you have Databricks, enter your identifier and click Connect.
  7. Choose which token type you will generate.
  8. Click Generate Token.
    Note: The token will disappear after one hour or if you navigate away from the page. To generate a new token, return to Step 1. 
  9. The next section of this guide will help you connect to your downstream solution.

Connect to Power BI

Connect With Analytics Models

Download Credentials File

  1. Create a file called 'config.share.
  2. Add the fields below:
    {
        "shareCredentialsVersion": 1,
        "bearerToken": "",
        "endpoint": "",
        "expirationTime": ""
    }
  3. Add the bearerToken, endpoint, shareCredentialsVersion and expirationTime values received from Procore to the config.share file.

Run user_exp.py script

You can use the following scripts to create a config.yaml file with the necessary configurations. 

  • For Azure Storage:
    cron_job: #true/false
    run_as: #pyspark/python
    source_config:
      config_path: #path to the config.share file
      tables:
      - '' # table name if you want to download a specific table. Leave it empty if you want to download all tables
    source_type: delta_share
    target_config:
      auth_type: service_principal
      client_id: #client_id
      secret_id: #secret_id
      storage_account: #storage-account name
      storage_path: #<container>@<storage-account>.dfs.core.windows.net/<directory>
      tenant_id: #tenant_id
    target_type: azure_storage
  • For MSSQL DB:
    cron_job: #true/false
    run_as: #pyspark/python
    source_config:
      config_path: #path to the config.share file
      tables:
        - '' # table name if you want to download a specific table. Leave it empty if you want to download all tables
    source_type: delta_share
    target_config:
      database: #target database
      host: #target hostname:port
      password: #password
      schema: #target schema (default to procore_analytics)
      username: #username
    target_type: sql_server

Run as PySpark

If your environment is already set up with Spark, choose the 'pyspark' option when requested or once the 'config.yaml' is generated, you can run the following commands to download the reports to the data directory.

  • For Writing to ADLS Gen2 Storage:
    spark-submit --packages io.delta:delta-sharing-spark_2.12:3.1.0,org.apache.hadoop:hadoop-azure:3.4.0,com.microsoft.azure:azure-storage:8.6.6,org.apache.hadoop:hadoop-common:3.4.0 --exclude-packages com.sun.xml.bind:jaxb-impl delta_share_to_sql_spark.py
  • For Writing to MSSQL DB:
    spark-submit --packages io.delta:delta-sharing-spark_2.12:3.1.0 --jars <Location of mssql-jdbc jar> delta_share_to_sql_spark.py

Run as Python

  1. From the command line, navigate to the folder by entering “cd <path to the folder>” command.
  2. Install required packages using “pip install -r requirements.txt” or “python -m pip install -r requirements.txt”.
  3. Open SSIS and create a new project.
  4. From the SSIS Toolbox drag and drop Execute Process Task.
  5. Double click Execute Process Task.
  6. Go to the Process tab.
  7. Next to Executable, enter the path to python.exe in the Python installation folder.
  8. In WorkingDirectory, enter the path to the folder containing the script you want to execute (without the script file name).
  9. In Arguments, enter the name of the script delta_share_to_azure_panda.py you want to execute with the .py extension and click Save.
  10. Click Start in the top ribbon menu.
  11. During the execution of the task, the output of the Python console is displayed in the external console window.
  12. Once the task is done it will display a checkmark.

Choose Your Own Method

Delta Sharing is an open protocol for secure data sharing. You can find the public GitHub repository for Delta Sharing at https://github.com/delta-io/delta-sharing. The repository includes examples and documentation for accessing shared data using various languages such as Python and Spark Connector (SQL, Python, Scala, Java, R).

 Note
Make sure you have appropriate permissions and access rights to download the required files and run Docker containers on your system. Always follow security best practices and guidelines provided by Procore when handling sensitive data and credentials.

Connect to Databricks

  1. Log in to your Databricks environment.
  2. Click Catalog.
  3. Under Delta Sharing, select Shared with me.
  4. Click Sharing Identifer to copy and share with the Procore support team.
    Note: When the sharing identifier is received, Procore will add it to the system. 
  5. Under Providers, click Procore's identifier.
  6. Click Create Catalog.
  7. Enter your preferred name for the shared catalog.
  8. Click Create.
    Your shared catalog and tables will now show under the provided name in the Catalog Explorer. 

Migrate an Existing Report

  1. Open the report you want to migrate.
  2. Open the Query Editor.
  3. Click New Source then click More. 
    Note: The values you are given will not be the same as the values in the examples below.
  4. In the search bar, type 'Delta Sharing'.
  5. Select Delta Sharing, then click Connect.
  6. Type or paste the Delta Sharing Server URL you received from the Procore. 
  7. Select the table you want to migrate.
  8. In the new query, click Advanced Editor.
  9. Copy the contents before the word 'in'. This will include the 'Source' and 'Navigation' steps from the query.
    Note: The screenshot below is just an example. Your values will be different.  
    source-and-navigation-query.png
  10. Click Done.
  11. Open the 'Company'.
  12. Replace the 'Source' and 'Navigation' steps with the ones copied from Step 9.
    Note: Make sure you paste the text before the word 'in'.
  13. Click Done. 
  14. Add a comma at the end of your text if there are additional applied steps.
  15. Look through the query for errors that may occur. For example, some columns do not exist in the latest version of Procore Analytics so we can remove these from Applied Steps.
  16. Next, you will remap the project's table.
  17. Open Projects Query.
  18. Click Advanced Editor.
  19. Paste the same connection code from above (copied from companies query) to the first two lines of the project query (before any steps starting with #).
  20. The new query now looks like the screenshot below.
    project-query.png
  21. Add a comma and change the code to point to a different table, like the example below:
    Example:

    Source = DeltaSharing.Contents("https://nvirginia.cloud.databricks.c...d-4692f841b2ca",
    [rowLimitHint=null]),pa_internal_test = Source{[Name="pa_internal_test"]}[Data],
    public_company_only = pa_internal_test{[Name="public_company_only"]}[Data],
    companies1 = public_company_only{[Name="companies"]}[Data],

    Change -> companies1 = public_company_only{[Name="companies"]}[Data],
    To ->   Projects = public_company_only{[Name="projects"]}[Data],
     
  22. “[Name=”projects”]” is what points to the correct table in Analytics 2.0.

Rename Report Tables and Columns

  1. Open the Query Editor for the report you need to rename.
  2. Find the column that is not in Procore Analytics 2.0.
  3. In the formula bar, replace the Analytics 1.0 column with the new one for Analytics 2.0.
  4. Another method would be to add a step to rename the columns from Analytics 2.0 to the values you had in your previous report. 
  5. For example, in Analytics 1.0, project_stage_name is now stage_name.
  6. If you go to the navigation step, you can rename the column here and fix errors in subsequent steps.
  7. Click Insert after making all updates. 
  8. Repeat this process for any other renamed columns.
  9. When you return to the error, you will see the issue resolved. 

Connect to Power BI Desktop

  1. Open your Power BI Desktop.
  2. From the Home page, click New to expand the section.
  3. Click Report.
  4. Click Get data from another source. 
  5. In the search bar, type 'Delta Sharing'.
  6. Select Delta Sharing, then click Connect. 
  7. Type or paste the Delta Sharing Server URL you received from Procore. 
  8. If this is the first time you are connecting to this source, you will be prompted to provide your Delta Sharing Bearer Token.
  9. After authentication, select the Procore Analytics tables you want to bring into your Power BI report. 
  10. Select Load to view your report or select Transform Data to make more transformations in Power Query. 

Set Up Row-Level Security

  1. Configure 'Project' and 'ProjectUser' Power BI Relationship
  2. Create a New RLS Role
  3. Add Users to a RLS Role

Configure 'projects' and 'project_users' Power BI Relationship

Configuring a Power BI relationship between the projects and project_users tables connects the data from these tables. See Microsoft's Create and manage relationships in Power BI Desktop icon-external-link.png. Two options for configuring this relationship are outlined below.

Option 1

  1. Open a Procore Analytics report in Power BI Desktop.
  2. On the Home tab, click Manage Relationships.
  3. In the 'Manage relationships' window, scroll through the options and select project_users (project_id) and click Edit.
    manage-relationships-analytics.png
  4. Complete the following in the 'Edit relationship' window:
    • Under 'Cardinality', select Many to one (*:1) and mark the 'Make this relationship active' checkbox. These options should be selected by default.
    • Under 'Cross filter direction', select Both and mark the 'Apply security filter in both directions' checkbox.
      edit-relationship-analytics.png
  5. Click OK to close the 'Edit relationship' window.
  6. Click Close to close the 'Manage relationships' window.
  7. Save the report.
  8. Continue with the steps in Create a New RLS Role.

Option 2

  1. Open a Procore Analytics report in Power BI Desktop.
  2. Click the Model view.


     
  3. Check the relationship between the 'projects' and 'project_users' tables. The relationship should have a (1) next to the 'projects' table and an asterisk (*) next to the 'project_users' table.
     Tip
    To only view the relationship betweeen the 'projects' and 'project_users' tables (as shown below), click the plus icon (+) next to the 'All tables' tab to create a new layout and add the 'Project' and 'ProjectUser' tables to the layout by dragging and dropping them from the Properties > Fields menu into the gray space.
    relationship-models-analytics.png
  4. Select the relationship by double-clicking on the connecting line between them.
  5. Complete the following in the 'Edit relationship' window:
    • Under 'Cardinality', select Many to one (*:1) and mark the 'Make this relationship active' checkbox. These options should be selected by default.
    • Under 'Cross filter direction', select Both and mark the 'Apply security filter in both directions' checkbox.

      edit-relationship-analytics2.png
  6. Click OK to close the 'Edit relationship' window.
  7. Save the report.
  8. Continue with the steps in Create a New RLS Role.

Create a New RLS Role

  1. On the 'Modeling' tab, click Manage Roles.
  2. In the 'Manage roles' window, complete the following in each column:
    • In the 'Roles' column, click Create and enter a name for the role in the 'New role' field. Project User is the role name used in the image below.
    • In the 'Tables' column, select project_users.
    • Select Switch to DAX editor.
    • In the 'Table filter DAX expression' column, enter [email_address] = userprincipalname().
      security-roles-analytics.png
  3. Click Save in the 'Manage security roles' window.
  4. Save and publish the report.
  5. Continue with the steps in Add Users to a RLS Role.

Add Users to a RLS Role

Once the RLS table relationship is configured, users must be added to RLS roles in the Power BI service in order to view a Procore Analytics report. For more information, see Microsoft's Row-level security (RLS) with Power BI: Working with members icon-external-link.png.

  1. Open a Procore Analytics report in the Power BI service.
  2. Under 'Datasets', click the ellipsis (...) next to the report you want to set up RLS for and click Security.
  3. Select the role you want to add one or more users to.
     Tip
    Using Office 365 Distribution Groups can simplify the process of adding multiple users to an RLS role at one time. See Microsoft's Get started with Microsoft 365 Groups in Outlook icon-external-link.png.
  4. Enter their email addresses and click Add.
  5. Click Save.