Well, no.

Of course. Obviously, when your data are changing and you want to get all of the latest updates in your viz, you’ll want to use a live query. But if that’s not the case, then an extract is clearly better, especially with Hyper in 10.5, right?

Well, no!

Shoot! This is complicated? When will live beat an extract? Let’s take a look at a few cases.

A Few Basics

To understand what’s going on, you should have a basic understanding of how live and extracted data sources are used by the system. If you feel a bit shaky here, I’d recommend my previous post on live vs extracts. But in a nutshell:

• When you’re using an extract, the query defined by the data source is run and the whole resulting table is persisted in either a TDE (in Tableau 10.4 or before) or a Hyper database (in 10.5 and later). The queries produced by your workbook are then run against this table.
• When you’re running live, the queries from your workbook are composed with the data source query. In simple cases, at least, this will result in a single query that is pushed down to the target database system, and only the results needed for the viz are returned.

We’re going to look at a few cases where live can do better than an extract. As we look at them, pay particular attention to:

• The time to run the remote query,
• The time to transfer the data, and
• The time to run the local query.

These aren’t rigorous perf numbers, but to give you a sense of scale, here’s my setup:

• Tableau 10.5 (with Hyper) running on a i5-2500 with 8GB of RAM.
• SQL Server 2017 Express Edition running on an i7-3770 with 16GB of RAM.
• All wired together over gigabit Ethernet.

So nothing too grand. In any case, the lessons here should carry over to other hardware.

The data set is a stock history set from Kaggle that records daily stats for large number of stocks and ETFs. The schema looks like:

history(ticker, type, date, open, high, low, close, volume, openInt)

Loaded into SQL Server and indexed on (ticker, date), this results in 17.4M rows and about 1.5GB of storage. (I have no idea what the provenance or accuracy of these data are, but for this work only the size is relevant.)

Let’s try to beat an extract!

Nail The Index

Let’s start with an easy case: let’s find the yearly average close for Tableau’s stock. I’ll drag the ticker into filters, years into columns, and Avg(Close) into rows. It’s an award-worthy viz:

This is also an almost ideal query for our SQL Server database: it makes excellent use of the index, so the query is exceptionally fast to run; and because the aggregation happens remotely, there are almost no results to send over the wire. By looking in the log, I find that it takes a whole 0.006 seconds to run this query and fetch the results. How can we possibly beat that?

Indeed, if we recreate the same viz with an extract, Hyper takes more like 0.2 seconds to compute the viz.

So SQL Server is faster than Hyper? Well, in this case it is, but we’ve almost cheated by practically tuning it to answer this query quickly. Hyper, on the other hand, doesn’t require (and doesn’t allow) us to tune its setup. So we’re comparing the best case for SQL Server to a case for Hyper.

But the lesson is still sound: if your query (a) lines up well with the setup of your remote database, and (b) transfers very little data, then we can actually beat a Hyper extract.

Be Truly Ad Hoc

Let’s try to avoid pandering to SQL Server quite so much and just ask for the number of records my data set has each year:

Now SQL Server takes a bit longer: 5.53 seconds. Trying this against the extract shows what Hyper can do: 0.193 seconds. In this case, both engines have to do roughly the same amount of work, but with it’s column-based, in-memory execution, Hyper is the clear winner!

Except that we haven’t taken into account the cost of generating the extract. When we refresh it, we find that it takes us 67.8 seconds to generate a 435MB extract. If we add that in, SQL Server starts looking pretty good:

Applying a little algebra, that means that to recoup the cost of our extract, we’d need to run our viz query a hair over 15 times. Often times this will be worth it, but if the query is truly one off, I’d rather spend 5.53 seconds than 68.

Blow Up the Extract

Let’s try something more horrible. Let’s say that in addition to the historical stock prices, we have a table of customer holdings. We’ll keep it simple; our customers have static holdings that look like:

customerholdings(customer, ticker, amount)

(I don’t actually have any customers, so I randomly generated 20 holdings for each of 20,000 imaginary customers.)

We want to do things like look at the total value of all customers’ holdings over time, so we join the holdings to the price history.

We then create a calc to compute the value each customer’s holdings and make a viz:

In case you’re interested, that giant spike is caused by a few odd stocks like DryShips Inc. (DRYS), which somehow peaked at $1,442,048,636.45 in 2007. I don’t comprehend. The graph looks funny, but again, this doesn’t matter for our analysis.

What we care about is that this query takes 133 seconds to run—it’s a fair bit of work for SQL Server to do. How about the extract?

Well, let’s do a little back of the envelope computation. If we execute the full join in SQL Server and don’t aggregate anything down, instead of the 17 million records in our history table, the result set will have about 441 million records. And these records are larger than the history rows because they have customer information as well.

Optimistically, this will end up being something like 10 gigabytes of data that I have to move over the wire, and store in a local extract. And that’s all before I even get to ask my query. So unless I’m doing this a lot, I’m simply not going to bother.

Wrapping Up

So we’ve seen a few cases where live queries may be preferable to extracts, leaving aside the obvious cases where you simply want the most current data.

One thing we didn’t talk about is federated queries: queries that span multiple data sources. As a general rule, federation makes extracts look better relative to live, because live starts to look worse. Live works best when the engine can push operations that reduce data volumes off to the remote system—operations like aggregations and filters—and federation tends to interfere with that pushdown.

But that’s another ball of wax. I’ll write more on federation soon.

But how do I get at those queries? I was talking with Yvan Fornes, and he suggested that I write about how I do it.

Challenge accepted! Except I may have gone overboard: in this post I’ll explore three ways to find the queries underlying your viz.

Setup

To illustrate things, I’ve built a very simple viz against a very simple table I have in SQL Server. Here’s the data source—twelve rows in all their glory.

And here’s the viz—I’m counting students by class:

With these, I’m going to show how to find out what query is issued when I refresh the view:

I’m going to walk through these with my setup, but you can do this with your own favorite viz just as well.

Method 1: Look to the Logs

Tableau logs all of the queries it issues, plus a lot of diagnostic information along the way. In theory, you could read look through these logs with a text editor, but you don’t want to do that.

Instead, head on over to GitHub, where you can find the Tableau Log Viewer. You can clone the project if you want to build it yourself, or grab a prebuilt copy from the releases folder like a normal human being.

Once you have this installed, go ahead and start it up. You should see a screen like this:

You can use this to read through historical logs, but there are a lot of events, and narrowing it down to what we care about can be hard. Instead, we’ll use Live mode and capture the events that result from our refresh as they happen.

So lets capture some logs! If you’ve pre-loaded your workbook in Tableau, the first thing to do with TLV is to open your logs, which should be in My Tableau Repository\Logs\log.txt. This will load a lot of noise, but once you have that open, you can:

1. Start live mode.
2. Clear out any history that you have, leaving you with a nice, blank starting state. Don’t worry: this will only clear what’s loaded in TLV, not the log file on disk.
3. Switch over to Tableau and refresh the data source.
4. Come back to the Log Viewer and turn off live mode.

Now we have a mess of log entries. It can be a bit overwhelming. But we’re interested in the queries we’re issuing, so let’s scan for a begin-query event…

Now we can right-click on it and ask to “Highlight all events of this type”.

This will make it easier to see the relevant events. Even so, there will likely be a few queries to wade through, but it shouldn’t be too overwhelming. You can double-click on an entry to see more details. As you hunt through the list, you may see a number of entries like this

that query the system for metadata needed to understand the server. But you should also have the query for your data source. In my case, this is the second begin-query element:

That’s it: if you’ve read my blog entry on dimensions and measures, you should recognize this as the query populating my simple viz.

Method 2: Tableau Performance Recorder

Another Tableau-provided tool is the Performance Recorder built right into Tableau. This has the advantage of being built-in and potentially simpler to use than the Log Viewer.

To use the Performance Recorder for our example, we open up our viz, and then:

1. Start the recording by clicking Help→Settings and Performance→Start Performance Recording.
2. Refresh the data source as before.
3. Stop the recording by clicking Help→Settings and Performance→Stop Performance Recording.

Once you stop the recording, Tableau will take a moment to put the report together, and then open up a Performance Recording viz:

In our case, there’s not much to see, because the query was so fast (a whole 12 rows!) and by default, the viz filters out events that take less than 0.1 seconds. To see what’s going on, we have to edit the filter so that we can see our query—let’s see everything:

Now we can see a set of query events. We can click on one to see its text. Even though it’s quick, the longest duration query is the one we want:

This was a simpler task to complete, but I find it easier to understand everything that’s going on using the Log Viewer. Ultimately, though, this is up to personal preference and the task at hand.

Method 3: Database-Specific Options

If you’ve been reading along, you may notice that in my posts I’ve used neither of the approaches above, and have instead used the SQL Server Profiler.

The primary advantage to the Profiler is familiarity—for me, anyway: I worked on SQL Server for a number of years, and I know its tools reasonably well. Profiler is also useful if you want to understand what’s happening on the database side of the conversation, but for most cases this is likely to be irrelevant.

The downside to Profiler is that it only works with SQL Server. If you have another database, you’ll have to learn whatever tools it happens to have. So I’d recommend using one of the alternatives above, but I’ll give a brief show-and-tell for completeness.

I’ll assume that Tableau is already open to your workbook. The first thing to do is to start Profiler and connect it to your SQL Server instance. Next, you want to start a trace—the defaults are good:

When you want to record your activity:

1. Clear the current trace in Profiler.
2. Switch over to Tableau and refresh the data source.
3. Switch back to Profiler and pause the trace.

Now you have the view of the world from SQL Server’s point of view. And as with the Log Viewer, there’s a lot here:

It can be a little hard to separate the wheat from the chaff. But after a little digging, we find our query buried in a sp_prepexec, and the whole statement starts with a declare:

Final Thoughts

There you are: more than you ever wanted to know about finding the queries underlying Tableau. Now maybe you can figure out what LOD calcs are really doing. If not, stay tuned and I’ll share—once I wrap my head around them properly.

But as it turns out, that’s the wrong question: Tableau doesn’t have a list of all the custom SQL syntax it supports because it really is just passing along the SQL code as you’ve typed it.

So why would a perfectly reasonable custom query fail? And what’s the link to SQL injection? Read on!

The Issue

To explore this, I’m going to use a really simple table and my favorite database system, SQL Server. Here’s the schema for the table:

students(name, class)

Now, if I put a very simple query into Tableau, I get exactly what I’d expect:

But if I try something wee bit more interesting, Tableau gives me an error:

What gives? How is Tableau screwing up my query? This works fine if I run it directly against SQL Server:

As it happens, the error doesn’t come from Tableau: it comes from SQL Server—because the query is wrong.

What Gives?

When you put custom SQL into Tableau, Tableau passes it along nearly unadulterated to the target system. But by “nearly unadaulterated” I mean “wrapped in a subquery”.

So, for example, when I enter:

select class, count(*) 
from students 
group by class

What actually gets passed along to SQL Server is:

SELECT TOP 1 *
FROM (
    select class, count(*)
    from students
    group by class
) [Custom SQL Query]

You can see our query in there, but it isn’t issued by itself: it’s wrapped in an outer SELECT and lives on as a subquery creatively named “Custom SQL Query”.

I’ll come back to why this outer query is there in a second. But first, let’s take this whole query and try running it in SQL Server directly:

This error looks familiar: It’s telling us that the count(*) statement in the select needs to have a column name if it’s going to live in a subquery. If we fix this by giving the count a name…

FROM (
    select class, count(*) a_name
    from students
    group by class
) [Custom SQL Query]

…then the query will run correctly SQL Server—and the inner bit will work in Tableau:

One lesson here is that if you’re trying to debug why your query isn’t working in Tableau, you can wrap it as a subquery and try debugging it in the underlying database directly. Once you have it working in that context, it will probably work as custom SQL in Tableau.

Oh, and why the SELECT TOP 1 *? The first thing Tableau wants to do when it’s faced with custom SQL is to get the resulting schema, and fetching a single row is a good way to do this. If this succeeds, Tableau will use your query in other combinations, but always as a subquery.

A SQL Injection?

If all of this makes you think of SQL injection, you’re not crazy. A SQL injection attack works by letting someone provide code that gets naively splatted into a SQL statement that’s sent along to the database—and that’s a lot like what’s going on here.

Obligatory XKCD Reference

I’m sure that someone out there can do something a whole lot slicker (and more nefarious) than this simple example, but what if we put this garbage SQL into our custom SQL?

select 1 as foo) [Custom SQL Query]
create table SUPER_NEFARIOUS_TABLE (EVIL_COLUMN int)
select 1 from (select 1 as foo

This is clearly malformed SQL all by itself, but remember that it’s going to be inserted into another query:

SELECT TOP 1 *
FROM (
select 1 as foo) [Custom SQL Query]
create table SUPER_NEFARIOUS_TABLE (EVIL_COLUMN int)
select 1 from (select 1 as foo
) [Custom SQL Query]

Now this looks more like SQL. In fact, it’s three separate SQL commands:

SELECT TOP 1 *
FROM (
select 1 as foo) [Custom SQL Query]

create table SUPER_NEFARIOUS_TABLE (EVIL_COLUMN int)

select 1 from (select 1 as foo
) [Custom SQL Query]

What happens when Tableau executes this? It gets its data from the first query, but the second and third queries run as well, and number two is a little suspicious. That red bit will actually create a table on your target database—assuming you have the proper permissions—right from custom SQL.

Of course, here I’m just creating a table, but I could drop one just as easily.

Stay Safe

Here’s the good news: I’m just not that clever.

While I’ve injected some potentially nefarious SQL into my custom SQL, so far all I’ve really done is stage a SQL injection attack on myself.

Naturally, this would be harmful if I could get you to run the query. What if I sent you a present: a nice, simple, harmless, cuddly, nonthreatening .twb…

…with laser beams! Or better yet, nasty custom SQL.

But (alas!) I’m not the first one to figure out this potential attack vector, and some smart engineers at Tableau decided to warn you when custom SQL is present in your .twb. So when you get a suspicious workbook—any workbook with custom SQL—you’ll be confronted with a warning like this:

If you see this warning, you should take note and make sure you understand what that custom SQL is doing before you proceed.

And to be doubly safe, don’t accept any .twb from me.

In this post we’ll explore a common way to implement row-level security on top of a relational database and see why it may not be as secure as it looks.

A Pop Quiz

But before we get to the crux of the issue, here’s a quick quiz. I promise it’s relevant.

What will each of the following languages do when a is equal to 0​?

1. C, C++, C#, Java, and most other C-family languages:
   if (a != 0 && 1/a > 0) { /* Do something */ }
2. Pascal:
   IF a <> 0 AND 1/a > 0 THEN (* Do something *)
3. SQL:
   SELECT *
FROM T
WHERE a <> 0 AND 1/a > 0

Obviously, I’m asking about short circuiting behavior. I’ll let you ponder and reveal the answers in a moment. But first, back to row-level security.

The Setup

Imagine that we have a table of sensitive customer information:

customers:
id  ssn        balance
--- ---------- --------
1   123456789  150.00
2   234567890  250.00
3   345678901  350.00
4   456789012  450.00
5   567890123  550.00
6   678901234  650.00
7   789012345  750.00
8   890123456  850.00
9   901234567  950.00

(Apologies if I’ve exposed your SSN…)

We want to provide our employees access, but only to their customers, not the entire set.

A common way to do this on a system that doesn’t have built-in row-level security is to (a) add a security table that expresses which rows each user is allowed to see, (b) build a view that uses this security table to restrict the rows that each user sees, and (c) force everyone to access the data through the view.

So, I first create a security table that maps users to the customers they can see. E.g.,:

access:
uid    cid
------ -----
alice  1
alice  2
alice  3
isaac  4
isaac  5
isaac  6
bob    7
bob    8
bob    9

This means, e.g., that isaac should only be able to see customers numbered 4, 5, and 6. To enforce this we create a view:

CREATE VIEW sec_customers
AS
SELECT *
FROM CUSTOMERS
WHERE id IN 
    (SELECT cid FROM access WHERE uid = USER_NAME())

I’m showing this with SQL Server, so I’m using the built-in function USER_NAME() to dynamically modify the query based on the user who accesses the view. The specifics here will vary system-to-system, but you should be able to accomplish something similar.

We’ll restrict access to the base table, and let users only come in through the view. Now when isaac selects everything from sec_customers, all he sees is:

id  ssn        balance
--- ---------- --------
4   456789012  450.00
5   567890123  550.00
6   678901234  650.00

The Punchline

Pretty good! But before we celebrate, let’s look at the answers to our quiz:

1. In most C-family languages, compound predicates like this short circuit: the system will check whether a != 0 and only execute the 1/a > 0 bit if a isn’t zero. The body of our conditional won’t be executed, but life will go on as usual.
2. In Pascal there is no short circuiting: the system will always execute all of the parts of the compound predicate. So if a is zero, the system will throw a divide-by-zero error.
3. In a particularly awesome twist of semantics, SQL short circuits but doesn’t guarantee order of operations. So this code may execute fine if it tests a <> 0 first, or it may throw an exception if it tries the division first—you’re at the whim of the optimizer.

What does this have to do with row-level security? As I mentioned when discussing extract types in Tableau, when you write a query against a (virtual) view in SQL, your query is composed with the view query, and this whole thing is then optimized. But SQL doesn’t generally respect the order of the operations you’ve written down, and this disregard runs deep. There is no “query boundary” when you compose queries: your operations can get shuffled around anywhere in the plan.

And that’s a problem when it comes to security.

To illustrate, let’s try another query against my “secured” customer table. I’m going to guess the Social Security number of a customer that I shouldn’t have access to, and see what I can find with a little SQL.

If I guess incorrectly, everything works as we’d expect:

SELECT *
FROM sec_customers
WHERE 1/(ssn - 789012346) = 0

id  ssn        balance
--- ---------- --------
4   456789012   450.00
5   567890123   550.00
6   678901234   650.00

But if I guess correctly:

SELECT *
FROM sec_customers
WHERE id = 7 AND 1/(ssn - 789012345) = 0

id  ssn        balance
--- ---------- --------
4   456789012  450.00
5   567890123  550.00
6   678901234  650.00
Msg 8134, Level 16, State 1, Line 16
Divide by zero error encountered.

And now I know that a customer has an SSN of 789012345: I’ve leaked information that I shouldn’t have leaked. And with a little work, I may be able to narrow this down to a particular customer.

What happened? Looking at the query plan for this query it becomes more clear:

The syntax implies that customers I don’t have access to will be filtered out before they hit my query. But the optimizer has reordered the operations: the security filter is enforced by the join, and my predicate has been “pushed down” and folded into the table scan. The result is that the predicate sees the entire customer table, which results in an information-leaking exception.

Coda

I’ve shown this with SQL Server, but this isn’t a criticism of that system. Aside from the specifics around identifying the user, the mechanism I’ve shown is likely to apply to any database with an optimizer that can rearrange operations—which is to say any database worth it’s salt.

That said, while SQL Server did add first-class row-level security in SQL Server 2016, it’s likely that it’s using a similar mechanism under the hood. As Microsoft’s notes:

Carefully crafted queries: It is possible to cause information leakage through the use of carefully crafted queries. For example, SELECT 1/(SALARY-100000) FROM PAYROLL WHERE NAME='John Doe' would let a malicious user know that John Doe’s salary is $100,000. Even though there is a security predicate in place to prevent a malicious user from directly querying other people’s salary, the user can determine when the query returns a divide-by-zero exception.

Is this a worry? Well, I suppose it depends. But I’d be reluctant to rely on this mechanism if I had a real concern about exposing data.

The impetus for this post is a number of statements I’ve seen along the lines of:

A live data source is just a real-time extract of your data.

This is my favorite kind of wrong: subtly wrong.

We’ll come back to extracts shortly, but first I want to take a digression through views. That will be our database perspective for the day.

The Database Perspective

For the purpose of this example, we’re going to imagine that we have two super-creative tables named products and sales. For this post, we’re not going to care much about the data, but let’s assume a schema for each of these tables:

products(pid, description, price)
sales(pid, customer, count)

Here, pid is the product identifier—and the join key between the two tables.

We might want to ask questions of these data like, “how much did each customer spend?” If we were going to jump right in and use these in Tableau, we’d start by creating a new data source that joins these tables together. After joining the tables, we’d also create a new column with the total spend for each sale.

But we’re going to stick to database-land, so we’ll write this data source as a SQL query:

SELECT products.pid, description, price, customer, count,
        count * price AS total_sale
FROM products JOIN sales ON products.pid = sales.pid

To use this in a viz, we need to be able to run other queries against the results of this query. To do that, we need to create a view , which caches the query and gives it a name that we can write other queries against:

CREATE VIEW full_sales
SELECT products.pid, description, price, customer, count,
        count * price AS total_sale 
FROM products JOIN sales ON products.pid = sales.pid

Going forward, we’ll call this our “data-source query”.

Now we could use this in Tableau to create a viz that looked at total sales per customer. Using what we learned last time, we can also write the query that our viz will generate as:

SELECT customer, SUM(total_sale)
FROM full_sales
GROUP BY customer

We’ll call this our “viz query”.

So far this has all been a long preface for the real meat of this post. We want to understand what exactly the database engine does with this last query. And the answer depends a lot on what we actually mean mean by “caching a query”.

The Meat of the Post

When we said we’d cache a query, we could have meant two very different things. The simplest is that we could cache the result of the query. I.e., when we create the view, we could effectively create a new table called full_sales, run the data-source query, and populate full_sales with the result.

This is sometimes called a materialized view , and makes it pretty simple to understand what our viz query does: it just runs against the data in the table we’ve cached.

But there’s another possibility: we could cache the query itself. I.e., when we create the view, we won’t actually run the query at all; we’ll just stash the SQL and use it to produce results on demand later.

This is generally called a virtual view, and it would be pretty uninteresting if we just reran the data-source query every time we used it. But that’s not what databases do.

Instead, when we run our viz query against a virtual view, the database engine will compose the two queries together: it will effectively do an internal rewrite of the query so that it looks something like:

SELECT customer, SUM(total_sale) 
FROM (
        SELECT products.pid, description, price, customer,
                count, count * price AS total_sale 
        FROM products JOIN sales s
            ON products.pid = sales.pid
        ) AS temp
GROUP BY customer

The full_sales reference has just been rewritten with the subquery that defined it.

There are a few relatively obvious trade offs between these approaches:

• A materialized view can take a lot of space, depending on how much data the query produces.
• A virtual view takes almost no space at all, since all it stores is the query.
• A materialized view may take a while to build, but can ultimately save time if it precomputes data that speed up the queries that are run against it.
• A virtual view takes almost no time to create, but also doesn’t precompute anything.

But this misses the main event: in many cases, the fully-composed query will be much faster to execute than each half individually.

There are two big reasons for this. First, because the database system gets the whole, composed query at once, it can optimize the whole thing globally. To give a simple example, assume that the viz query filtered out a lot of the underlying data—maybe the viz only shows information on a single customer. With separate view and viz queries, the data-source query has to produce data for all the customers, just to have them filtered out by the view query. By considering them together, that filter can be pushed down and executed early, short-cutting a lot of the work.

Second, assume that the data-source query was executed on one system—say a SQL Server database—and the viz query was executed some where else—like Tableau. If the data-source query produces a lot of data, all of it needs to be sent over the network. If the queries can be composed and the whole thing can be executed by the source database, then only the final viz query results need to be sent. And through aggregation, Tableau viz queries usually reduce the data substantially.

For these reasons, running the viz query on top of the virtual view may perform significantly better than taking an extract and then running the viz query separately.

Bringing This Back to Tableau

Let’s connect this back to Tableau. But with the heavy lifting complete, the connection is straightforward:

• An extracted data source is a materialized view: the data-source query is run to pull data from your database, and the data are materialized into a TDE or (with Tableau 10.5 or higher) a Hyper database. View queries then run against this materialized extract.
• A live data source is a virtual view: the data source just stores the query, and Tableau composes it on the fly with whatever query your viz generates. This composed query is issued in one shot to your database.

Done. But now you can see how the explanation at the top is subtly wrong.

A live data source is just a real-time extract of your data.

If this were true, then every time you modified or refreshed your workbook, the system would first run the data-source query and suck down the results, and then run the viz query to aggregate it all down. But this striation doesn’t happen: with a live data source, the whole view+viz query is composed, and this whole query is then executed. Every time.

In the right circumstances, this can mean the query that’s run is more optimized and flows much less data down from the database. And that can perform very well.

Other times, the cost to materialize the extract is worth it because the construction cost is amortized over a whole lot of viz queries, all of which benefit from the extract. But that would be for naught if the extract were re-created for each viz query.

Coda

After all of this, it’s worth popping up to recognize the obvious benefit to live data sources: liveliness. Since live data sources go directly against your database, they see the changes to your data as those changes occur. This can be valuable unto itself, regardless of the performance trade-offs.

I don’t recall the precise phrasing, but it went something like this:

Dimensions are usually those fields that cannot be aggregated; measures, as its [sic] name suggests, are those fields that can be measured, aggregated, or used for mathematical operations.

Or this:

Measures are the result of a business process event… Dimensions are reference variables that give context to measures.

I don’t really mean to criticize these definitions, but to a database guy, they seem rather imprecise. For someone with a little SQL know-how, the actual definition is both crisp and helpful in understanding what Tableau really does under the covers—this helps predict what actions in the UI will do, so you don’t just blindly drag-and-drop until things look right.

The rest of this post is a crisp explanation of dimensions and measures for someone who knows a little SQL.

Tableau does a lot of things, but at its core—or what I like to think of as its core—it’s an aggregation engine: you tell it how to slice-and-dice your data and what aggregates to apply, and it does so. Then there is some visualization on top. That’s nice, too.

What this means is that so long as we confine ourselves to “simple” things, and our data are in a table T, then Tableau is going to produce a query that looks like:

SELECT some_stuff
FROM T
GROUP BY some_other_stuff

When I say “simple”, I mean that we’re going to confine ourselves to dragging fields into the Columns, Rows, and Marks shelves:

To illustrate, I swiped some highly confidential student records from the nearby college and dumped them into SQL Server. Here they are:

And now the meat of the post: Tableau’s rule for generating a SQL query from the collection of items on each of the shelves. It’s very simple—don’t blink:

• If a field is used as a dimension, then it’s added to the GROUP BY and SELECT clauses.
• If a field is used as a measure, then it’s only added to the SELECT clause, with the appropriate aggregation applied.

That’s it. The order doesn’t matter. The exact shelf doesn’t matter. All we care about is whether each field is a dimension or a measure.

Let’s try this out. We’ll just use the rows shelf, and drag out both Class and Name as dimensions. According to our rules, we should then generate:

SELECT class, name
FROM students
GROUP BY class, name

Let’s see what happens. First, here’s what things look like in Tableau:

This looks promising. Now let’s look at the query that was generated. There are various ways to do this, including digging through Tableau’s logs with the Tableau Log Viewer. I’m going to use the SQL Server Profiler to get the query.

I won’t give all the Profiler details. Tableau issues a bunch of little metadata queries, but if we dig a tiny bit, we find the main course:

SELECT [students].[class] AS [class], 
       [students].[name] AS [name]
FROM [dbo].[students] [students]
GROUP BY [students].[class], [students].[name]

And there we have it. A few more characters, but exactly the query we predicted. I’ve color coded these examples to make it easier to line up the fields.

But what are those “Abc"s? Well, Tableau wants to put a mark for each value that it’s calculating, and we haven’t told it what data to show, so it uses “Abc” as a placeholder. We can improve our viz a little and get rid of these by moving the Name to “Text” shelf:

A visualization guru might scoff at this. But I’m not a visualization guru. What we care about is the query this generates. And although the visualization has changed, the underlying query doesn’t: we’re still using both Name and Class as dimensions, and each unique pairing of these values shows up once in the viz.

Let’s try to use this knowledge to direct the viz we get. Let’s say that we want to know how many classes each student is taking. In SQL, we might write:

SELECT COUNT(class), name
FROM students
GROUP BY name

How do we get this in Tableau? Reviewing our rules, we see that we should leave Name a dimension, and make Class a measure with COUNT as its aggregation.

Let’s try it. Here I’ve put Name on Rows, and COUNT(Class) on Text:

That looks right: we have one mark for each distinct name, that mark is the count of classes. Just to confirm, let’s go to the SQL:

SELECT COUNT_BIG([students].[class]) AS [cnt:class:ok],
       [students].[name] AS [name]
FROM [dbo].[students] [students]
GROUP BY [students].[name]

Spot on!

Again, we can make a better viz—say, a bar chart—but rest assured, it’s just another way of presenting the same data from the same query:

(I always knew Elsa was an overachiever.)

To recap, dimensions and measures are really very simple. Dimensions are the things that you group by; they show up in both the GROUP BY and SELECT clauses of the underlying query. And measures are the things you’re aggregating; they never show up in the GROUP BY clause, only aggregated in the SELECT clause.

Cheers,
-Isaac