Why You Need Rock's BI Tools

With all of the reporting tools in Rock, why do I even need the BI capabilities? That’s a great question! It really comes down to speed and simplicity. Think about your kitchen. It’s nice and neat with everything having its place (even if yours isn’t, pretend it is). For your normal meals, this is great and organized, nothing is ever lost. That’s Rock’s normal mode (we’d say “transactional mode”).

Now let’s consider a Thanksgiving feast for thirty people. To speed up the cooking you’re probably going to pull everything you need out of the cupboards and line it up in the order you’ll need it. As you start wanting to run large reports and find insights into your data, Rock needs to do the same thing to stay efficient. It needs to arrange the data in a different fashion that’s optimized for speed.

Most systems don’t do both the daily cooking and the large meal. They require you to manually design the process for large analytics tasks. Rock’s BI tools simplify this process and automate much of it. You still need to provide it with a little guidance on what you’d like to see, but it’s much easier than traditional systems.

As a part of this process, Rock will create a set of new tables (aka, models) that contain a simplified version of your data that is very fast to process. These tables will need to be updated on a routine basis. We’ll discuss how you set that up a little later. It’s important for you to know, however, that the BI Analytics models are not updated in real-time. They represent a snapshot of the data from the last time the analytics jobs ran (typically nightly).

Facts & Dimensions

Facts are the things you’re going to report on, such as Attendance or Financial Transactions. Dimensions (often referred to simply as Dim) are characteristics of the Facts. For instance, Dimensions of Attendance would be the Location, Date or Person of an attendance record. Dimensions of a Financial Transaction would be the Batch, Transaction Type or Account.

Another way to think of this is Facts are what you’re measuring while Dimensions are what you’ll filter by.

The tables that you will be importing into your BI tool are all SQL Server Views and start with the prefix "Analytics". For example, the fact model for financial transaction would be (AnalyticsFactFinancialTransaction). The dimension model for the account would be (AnalyticsDimFinancialAccount), the transaction type would be (AnalyticsDimFinacialTransactionType), and the date would be (AnalyticsDimFinancialTransactionDate), to name a few.

When displaying these models as a diagram the fact model usually goes in the center with the dimension models around it. This arrangement creates what is known as a Star Topology. Diagrams for Rock’s financial and attendance models can be found below.

• Tables in red are fact tables.
• Tables in blue are dimension tables.
• Tables with the calendar icon are date dimension. They contain several columns to act as filters (aka, slicers).

Analytics Source Date Table

The Analytics Source Date table ships with Rock and contains a row for every date from about 100 years ago through to about 100 years from now. So whatever date you're looking for, there should be a row in the table for it. Using the different properties of the Analytics Source Date table provides ways to approach dates and date-related data that would be much more complicated, and less performant, if you had to do all these calculations yourself. It's a powerful tool that you'll find yourself using more as you get more familiar with it.

Many properties for each date are listed in the table. One that you'll want to be aware of is the DateKey property, which can be used to link the Analytics Source Date table to a variety of other tables throughout Rock. For instance, the MetricValue table has a property called MetricValueDateKey which provides the date associated with a metric value in a format that can be used to join to the Analytics Source Date table's DateKey property.

Besides MetricValue, other tables that have a Date Key property you can use to link to Analytics Source Date include:

• Analytics Tables
• AttendanceOccurrence
• BenevolenceRequest
• Communication
• ConnectionRequest
• FinancialPledge
• FinancialTransaction
• Interaction
• Registration
• Step

Aside from DateKey, you'll find many other useful properties in the AnalyticsSourceDate table. For instance, you can get different formats of the date. You can also see the day of the week that the date is (e.g., '0' for Sunday, '1' for Monday). For each date you can also get the associated Sunday Date, which month the date is in for giving, as well as a variety of Calendar and Fiscal Year data showing things like which quarter or week the date falls in. There are also indicators telling you if the date is a holiday, or if the date falls within the week of a holiday.

The last property of the Analytics Source Date table is called Count. Every date in the table has the same value of '1' for this property. This lets you easily do counts against the dates. For instance, if you have two different dates then you could get all the records between those two dates and simply sum up the Count property to give you the number of days between the two dates.

For additional information on the Analytics Source Date table and its properties, check out this video.

Using Week Of Year

Providing year over year comparisons of metrics can be very helpful. Unfortunately, it can also be difficult and taxing on the server to implement. To help ease the creation of these metrics and reduce the processing load on the server you can use the WeekOfYear column on the AnalyticsSourceDate model. This field represents the week number of the day within the year for a Monday to Sunday week (just like the name suggests).

Since dates are incredibly difficult to standardize, there are a few points you should understand about how this field is calculated.

1. Date processing is very difficult due to the non-standard nature of time units (7 days in a week and 365.25 days in a year). While there are several 'standardized' (e.g., ISO 8601) ways of calculating the week number, none of them are <cough> standards. Each has strengths and weaknesses. The calculation described below seeks to provide a standard that works well for churches. While it's not perfect for all use-cases, we feel that it's very helpful when comparing weekly metrics, especially across years (comparing this week to the 'same' week last year).
2. For the sake of comparison, we wanted to ensure that every week number represents a full week.
3. The week of the year is calculated for the Sunday Date and then applied to every day in the week. Keep in mind that Sunday is typically the last day of the week (this can be adjusted in Rock). That means that days at the end of the year could apply to week one of the next year if their Sunday is in the new year.
4. There will be some years with 53 weeks in them. This happens roughly every 5.64 years (when the first of the year is a Sunday or Saturday on a leap year). This week won't have a previous year’s week to compare to. It's up to you how this is addressed in your dashboard. You can choose to show it as an outlier or filter it out.
5. Because of the fact that a week of the year can span two different years, it's important that your date filtering not use the Date property, but instead the SundayDate property. To help with this, use the SundayDateYear property to filter by year. Note that if for some reason you really don't want to include the data from previous years you could use CalendarYear. This will mean, though, that the first month/week of the year might not be a full seven days.

There is also a property called WeekCounter that is the week number for all time (at least since 1/1/1870 the first date in the table). This field is helpful to key off of if you want to compare this week to the last x weeks using SQL windowing functions. 

Historical Tables

As noted, these tables track an entity over time. To do that, a single entity (like a person) may have several records in the historical table, one for each change that was made. A couple of key fields, listed below, help to keep track of these records.

• EffectiveDate – This is the date that the record started. If it’s the first and only record for a person this date will be the date the analytics tables were first loaded. If the record is a subsequent change, it will be the date of the change.
• ExpireDate – This is the date when the record was no longer the current (latest) change. If this record is the current record the date will be 1/1/9999 (which helps you with SQL compares).
• CurrentRowIndicator – This column notes the latest row for the entity.

Current Tables

The records in these tables represent the most recent record in the Historical tables. The implementation of this is a simple view that looks for records where the CurrentRowIndicator = 1. Elegant, right?

Person Attributes

You can determine which person attributes you would like to have added to the BI models. You can also determine which are marked to track history. Tracking history allows you to see the changes to attributes over time.

Family Attributes

Like Person attributes, you can also mark specific family attributes to track their changes.

Calendars

Part of what makes the BI models so fast is how they handle dates. Each day has a row in a large date table. Along with the actual date is stored the year, month, quarter, etc. This allows the data to be filtered, or sliced, very quickly. Because some of the date information centers around fiscal years, which can be different for each organization, Rock allows you to customize the calendar dimension settings. You can configure this under Admin Tools > System Settings > Calendar Dimension Settings.

Keep in mind that changing Calendar Dimension Settings can be risky. For instance, changing the Start Date to a date in 2020 means nobody born before 2020 will show up in a data view looking for people by age.

BI Job

Rock’s Process BI Analytics job keeps the data that the BI tool uses up to date. The job is located in the Jobs Administration screen (Admin Tools > System Settings > Jobs Administration). While the job ships with Rock, you need to schedule it and run it at least once for the data to be available. (If you open your BI tools and don’t see any data, this is why.)

Let’s take a look at the BI job settings and what each does.

The BI job runs a number of processes, including:

• Process Person BI Analytics – Keeps person tables up to date and processes historical data.
• Process Family BI Analytics – Keeps family tables up to date and processes historical data.
• Process Campus BI Analytics – Keeps campus information in sync.
• Process Financial Transaction BI Analytics – Processes financial transactions and adds any new transactions.
• Process Attendance BI Analytics – Updates attendance data.
• Refresh Power BI Account Tokens – Keeps account tokens from expiring.
• Process Giving Unit BI Analytics - Updates giving unit data.

Account tokens must be refreshed every two weeks, otherwise they expire. If you view a report, the tokens are automatically refreshed. If you go more than two weeks without viewing a report, though, the BI job will refresh the tokens for you to keep them from expiring.

Now let's look at how you can customize the BI job by enabling and disabling these settings.

Enabling and Disabling BI Job Settings

You may wonder why you’d want to disable the BI job settings since they keep your data up to date. There are some valid reasons for disabling certain settings, though, such as if you decide not to use Power BI.

Keep in mind that the BI tables are only updated when the job runs; so, the tables aren’t 100% up-to-date every minute. You may want to update some data more often than others, running the analytics, say, four times a day as opposed to only once a day. For example, if you want to process person analytics more often than the others, you could disable the Process Person BI Analytics setting and create a separate Process BI Analytics job that runs the person analytics more often, separately from other analytics.

So, you can create multiple versions of the job to run different schedules. This gives you a lot of flexibility when it comes to processing data in a way that’s most useful to your organization.

On-Premises Data Gateway

As you use PowerBI in the cloud you’ll want to ensure that your data is always up to date. While you can do that manually using the PowerBI Desktop application, you can automate the process using the PowerBI On-Premises gateway. This is a free tool that will keep your data in Rock in sync with the PowerBI cloud. Below are steps for getting this tool configured. If you prefer a video tutorial, this YouTube video from Microsoft covers the steps in detail: https://www.youtube.com/watch?v=GwuRQhm241c.

1. Download the ‘On-premises data gateway installer’ to your Rock server from your PowerBI account (located under the Download link).
2. Run the installer and select the ‘On-premise data gateway’.
3. After the install, you’ll be asked to enter an email address that is assigned to your account. This will allow you to link the gateway to your account.
4. Once your gateway is linked, you’re ready to define a source to your Rock SQL Server. To do this follow these steps:
   1. Select the Settings option on the PowerBI web portal (the gear icon), then select ‘Manage Gateways’. You should see the gateway you just named and registered listed.
   2. Place your cursor over the gateway. This should display an ellipsis. Click the ellipsis and select ‘Add a datasource’.
   3. Give your new datasource a name and enter the configuration info needed for a SQL Server Data Source Type. (Tip: you might consider using the same database username and password that is defined in your website’s connection string).
5. Finally, create a new PowerBI project using the desktop application. Connect to the same SQL Server and select the Rock BI models. (Hint, they’re the tables and views that start with the word “Analytics”).

Considerations for Embedded Reports

Rock includes the awesome ability to embed Power BI reports right onto a page, which provides a great user experience without end users having to learn or use a special reporting client. However, this power comes at a cost. We have compiled the various options for utilizing Power BI reports in Rock and what Microsoft charges for these features below.