Power Query - Solution Book PDF
Power Query - Solution Book PDF
Table of Contents
Solutions to Common Date Problems
Calculating Ages in Years ..........................................................................................................................3
Calculating Time Served in Days................................................................................................................4
Calculating Days Till Warranty Expiration.................................................................................................5
Creating Dates from Multiple Columns .....................................................................................................6
Creating Dates with Missing Days .............................................................................................................7
Creating Dates Using Custom Columns .....................................................................................................8
Deriving Dates from Words .......................................................................................................................9
Correcting Date Formats from Other Regions .........................................................................................10
Create a Calendar Table From Start to End Date ....................................................................................11
Solutions to Common Time Calculations
Calculating Hours Worked (spanning days) ............................................................................................19
Create Date/Time from Separate Values ................................................................................................20
Helpful Power Query Tricks
The Hidden Table Method – Keep Your Source Formatting ....................................................................23
How to Refer Back to Previous Steps .......................................................................................................26
How to Adjust FILTER & Conditional Column Formulas – Use Dynamic References ...............................28
Create a Template With Power Query ....................................................................................................31
Solutions to Appending Data Challenges
Pivot Table from Multiple Excel Tables ...................................................................................................35
Combine Data From Multiple Workbooks ...............................................................................................40
Combine ALL Excel Files in a Folder .........................................................................................................42
Adjust Folder Path from Local to SharePoint Drive .................................................................................46
Combine All Sheets in a File .....................................................................................................................48
Consolidate Data From Multiple Sheets in the Current Workbook .........................................................52
How to Select File Path with Power Query Parameters ..........................................................................56
Manage the File Path from an Excel Cell .................................................................................................58
Import from Folder & Clean Headers ......................................................................................................61
How to Append All Sheets when Data Starts from Different Rows .........................................................63
Append Sheets and Get Field Name from a Cell Outside the Table ........................................................65
Unpivot & Consolidate Data from Multiple Sheets .................................................................................68
Solutions to Common Data Analysis Problems
How to Create Buckets for your Data ......................................................................................................73
Master Power Query Power Query – Solutions Guide
Microsoft Power Query enables Excel users to import data from a wide range of data sources
Excel into Excel for analytics and visualizations.
Starting with Excel 2016, Power Query capabilities are natively integrated and can be
found under the “Get & Transform” section of the Data tab in the Excel Desktop
ribbon.
Excel 2010 and 2013 users can also leverage Power Query by installing the Microsoft
Power Query for Excel add-in.
Microsoft Power Query enables data analysts and report authors to connect and transform data
Power BI as part of creating Power BI reports using Power BI Desktop.
Microsoft SQL Business Intelligence Developers can create Azure Analysis Services and SQL Server
Server Data Analysis Services tabular models using SQL Server Data Tools for Visual Studio. Within
Tools for this experience, users can leverage Power Query to access and reshape data as part
Visual Studio of defining tabular models.
Common Common Data Service lets you securely store and manage data that's used by
Data Service business applications. Data within Common Data Service is stored within a set of
entities. An entity is a set of records used to store data, similar to how a table stores
data within a database.
Common Data Service includes a base set of standard entities that cover typical
scenarios, but you can also create custom entities specific to your organization and
populate them with data using Power Query. App makers can then use Power Apps
to build rich applications using this data.
1. Select the Date column and convert to a Date data type (if necessary)
2. Select Add Column (tab) → From Date & Time (group) → Date → Age.
1. Select the Date column and convert to a Date data type (if necessary)
2. Select Add Column (tab) → From Date & Time (group) → Date → Age.
1. Select the Date column and convert to a Date data type (if necessary)
2. Select Add Column (tab) → From Date & Time (group) → Date → Age.
2. Select Add Column (tab) → From Text (group) → Merge Columns. Define the separator
(comma) and name the new column (“Date”).
2. Select Add Column (tab) → From Text (group) → Merge Columns. Define the separator
(comma) and name the new column (“First of Month”).
2. In the Custom Column dialog box, the new column will be named “Custom Date” and the
formula will be as follows:
= #date([Year],[Month],[Day])
or
= #date([Year],[Month],1)
NOTE: The second version of the formula can be used if you want all dates to be on the first of each
month.
1. Select the column of month names and click Add Column (tab) → From Text (group) → Format
→ Add Prefix.
2. Select the button next to the “Foreign Date” heading and select “Using Locale…”
3. In the Change Type with Locale dialog box, set the Data Type to “Date” and the Locale to the
choice that best represents the country the data originated, such as Germany (Germany), and
click OK.
• We have an Excel Table named “TDates” which contains a list of dates and prices that denote
each time a price change occurred.
• Create a comprehensive list of dates that start from the earliest date in the table through the
latest date in the table.
• Each date should have the price that applies to that date.
Solution Steps:
Load the Date to a Query
1. Select Data (tab) → Get & Transform (group) → Get Data → From Other Sources → Blank
Query.
We are delivered a list of dates from April 1, 2020 through April 30, 2020.
3. Select List Tools – Transform (tab) → Convert (group) → To Table to convert the list to a table.
6. Add a column to hold extracted years by selecting Add Column (tab) → From Date & Time →
Year → Year.
7. Add a column to hold extracted month names by selecting Add Column (tab) → From Date &
Time → Month → Name of Month.
1. Select Data (tab) → Get & Transform (group) → Get Data → From Other Sources → Blank
Query.
We are presented with an error stating that lists cannot be built from dates using the “..” syntax.
We are delivered a list of values that represent the dates from April 1, 2020 through April 30, 2020.
4. Select List Tools – Transform (tab) → Convert (group) → To Table to convert the list to a table.
4. For the Date column, select the Filter → Date Filters → Custom Filter.
5. In the Filter Rows dialog box, set the parameters as shown in the following illustration.
1. Select Data (tab) → Get & Transform (group) → Get Data → From Other Sources → Blank
Query.
={Number.From(MinMaxDates[Date]{0} ) .. Number.From(MinMaxDates[Date]{1} ) }
We are delivered a list of values that represent the dates from the earliest date in the data through the
latest date in the data.
3. Select List Tools – Transform (tab) → Convert (group) → To Table to convert the list to a table.
1. Merge this table with the TDates table by selecting Home (tab) → Combine (group) → Merge
Queries → Merge Queries as New.
2. In the Merge dialog box, set the parameters shown in the following illustration.
1. Select the End Time column, press CTRL, and select Start Time.
2. Select Add Column (tab) → From Date & Time (group) → Time → Subtract. Remember, the
result is returned as a Duration data type read as Days.Hours:Minutes:Seconds.
3. Select Time → “Hours” to get the time rounded down to the nearest hour. Select Time → Total
Hours to get a precise number of hours.
1. Select the Day column, press CTRL, and select the Time column.
2. Select Add Column (tab) → From Date & Time (group) → Time → Combine Date and Time.
• access to all the features of Excel Tables, such as the automatic inclusion of new data.
• the ability to use a named range as your data source.
• the ability to retain the formatting of the source data.
NOTE: Merged Cells are not allowed in Excel Tables. If you convert a plain table to an Excel Table, the
merge feature will be removed. We will restore the merged look with a different feature.
1. Select the data range and upgrade to an Excel Table. This can be done via:
2. In the Create Table dialog box, verify proper range selection and ensure the “My table has
headers” option is NOT selected.
Step 2: Adjusting the Excel Table Before Import into Power Query
The Excel Table requires a few modifications before we import it into Power Query.
4. Remove the automatically applied table layout by selecting Table Design (tab) → Table Styles
(group) → bottom scroll button → Clear.
5. Give the table a proper name, like “NiceTable”, by selecting Table Design (tab) → Properties
(group) → Table Name.
6. Hide the “bad” headers by selecting Table Design (tab) → Table Style Options (group) → Header
Row.
7. If you had any merged cells (i.e. titles), you can restore the merged look by selecting the cells you
wish to “merge”, press CTRL-1 to open the Format Cells dialog box, and select Alignment (tab)
→ Text Alignment (group) → Horizontal → Center Across Selection.
1. Click anywhere in the data and select Data (tab) → Get & Transform Data (group) → From
Table/Range.
5. If any unnecessary header promotions or data typing was automatically performed, remove
those steps from the Applied Steps list.
a. select Home (tab) → Reduce Rows (group) → Remove Rows → Remove Top Rows
b. in the Remove Top Rows dialog box, enter a “1” in the Number of Rows option
7. Promote the first row as to a header by selecting Home (tab) → Transform (group) → Use First
Row as Headers.
9. Repeat the category labels by selecting Transform (tab) → Any Column (group) → Fill → Down.
At this point, we can Close & Load this to a new table, a Pivot Table/Chart, or into our Data Model.
Solution Steps:
1. Click in the data and select Data (tab) → Get & Transform (group) → From Table/Range.
3. Update the “Changed Type” step to have the Date column set as a Date data type.
We want to calculate the average price across all values in the “Price $” column, but we are unable to
use the Average feature from Add Column → Statistics.
4. Select the “Price $” column and click Transform (tab) → From Number (group) → Statistics →
Average.
We want to retain this value, but we need to apply it to the data from the previous step.
6. To restore the data from the previous step, click the button to the left of the Formula Bar to
insert a new step at the end of the query steps list.
7. Enter the following formula to point to the previous step named “Changed Type”.
= #”Changed Type”
NOTE: If the step name does not contain spaces, you can eliminate the # and quotes.
= ChangedType
9. Create a Custom Column by selecting Add Column (tab) → General (tab) → Custom Column.
= [Price $] - CalculatedAverage
13. To determine the Sales Reps who sold below the Difference, filter the Difference column for
items below 0.
1. Click in the data and select Data (tab) → Get & Transform (group) → From Table/Range.
4. Select the “Price $” column and click Transform (tab) → From Number (group) → Statistics →
Maximum.
We want to retain this value, but we need to apply it to the data from the previous step.
6. To restore the data from the previous step, click the button to the left of the Formula Bar to
insert a new step at the end of the query steps list.
7. Enter the following formula to point to the previous step named “Changed Type”.
= #”Removed Columns”
NOTE: If the step name does not contain spaces, you can eliminate the # and quotes.
= RemovedColumns
At this point, we could create a custom column that calculates the difference between each “Price $”
and the result of the CalcMax step then use the FILTER option to reduce the list.
Since we do not need to retain the “Price $” in the final result, we will work with the FILTER feature
directly.
Since we can’t reference query steps in the Filter Rows dialog box, the “8” is going to act as a
placeholder.
NOTE: When you directly modify a step in the Formula Bar, if that step contained parameters that could
be updated via the gear icon , you will lose the ability to perform updates in this manner. To restore
the gear icon , remove the original modification, make the adjustments using the gear icon , then
replace the adjustments.
12. Remove the duplicates from the Sales Rep column (future-proofing step).
• Using a supplied list of names and dates, create a template that repeats the dates for each name
in the list.
• Each name should have the ability to enter a start date and end date for each day in the list of
dates.
Solution Steps:
1. Click a cell in the list of dates and select Data (tab) → Get & Transform (group) → From
Table/Range.
4. Name the query ProjectDates. Remember this name; it will be referenced towards the end of this
solution.
6. Click a cell in the list of names and select Data (tab) → Get & Transform (group) → From
Table/Range.
9. Bring in the dates with the list of names by selecting Add Column (tab) → Custom Column.
10. In the Custom Column dialog box, name the new column Dates.
We need to bring in the list of dates (an entire table) for each row in the list of names.
11. For the custom formula, enter the following formula (a reference to another query):
= ProjectDates
This generates a nested table for each row in the Names column.
13. To display the complete list of dates for each name, click the Expand Columns button in the
upper-right of the Dates column.
14. Deselect the “Use original column name as prefix” option and click OK.
16. Change the data type for the Date column to Date.
We need to remove the redundant name entries from the Name column. We have a constant that the
first date for each employee is 6/1/2020. We will create a Conditional Column that tests for the
presence of this day and repeats the Name if it is a match. If it is not a match, it will leave the cell
empty.
18. In the Add Conditional Column dialog box, enter the following parameters:
Parameter Entry
New column name Employee Name
Column Name Dates
Operator Equals
Value 6/1/2020
Output Names
Else null
20. Select the “Employee Name”, hold down CTRL, select the Dates column.
21. Right-click a selected column heading and select Remove Other Columns.
• Main table with sales (fact table) is loaded as an Excel Table named “TableSales”.
Solution Steps:
Creating the Table Relationships
1. Select the TableSales table and click Data (tab) → Data Tools (group) → Relationships.
3. In the Create Relationship dialog box, set the parameters shown in the following illustration.
5. In the Create Relationship dialog box, set the parameters shown in the following illustration.
If we open the Data Model, we now have three separate tables (separate sheets) displayed.
We can view a graphical representation of the table relationships by selecting the Diagram View.
6. Select the cell to place the Pivot Table and click Insert (tab) → Tables (group) → Pivot Table.
7. In the Create PivotTable dialog box, select Use this workbook’s Data Model and click OK.
We can see the three tables listed in the PivotTable Fields list.
8. Place the MasterCustomer → Customer Name field in the Rows drop zone.
9. Place the TableSales → Sales Value field in the Values drop zone.
12. Format the Sales Values with a comma style, no decimal places.
13. Select the cell to place the Pivot Table and click Insert (tab) → Tables (group) → Pivot Table.
14. In the Create PivotTable dialog box, select Use this workbook’s Data Model and click OK.
15. Place the MasterArticle → Article Description field in the Rows drop zone.
16. Place the TableSales → Quantity field in the Values drop zone.
19. Format the Sales Values with a comma style, no decimal places.
NOTE: In the PivotTable Fields list, you can hide fields you don’t wish to see (either to make the list less
visually cluttered or to hide fields from the user), by switching to the Data Model → Diagram View and
right-clicking on a field. This will allow you to use the “Hide from Client Tools” option on the selected
field.
To use this feature, open the PivotTable Options dialog box and select Tools & Filters (tab) → Filters
(group) → Include filtered items in totals. Also, consider displaying an asterisk for the total to indicate a
non-filtered sum.
If you need to append more than two query results, select “Three or more tables” and place the desired
queries in the “Tables to append” list (right).
4. If you know the files are perfectly identical and require no further transformations, select
Combine. Otherwise, select Transform Data.
5. If you select Transform Data, you are placed in the Power Query Editor where all file metadata is
displayed.
8. In the Combine Files dialog box, we are presented with the options(s) as to which sheets or
tables we want from each file in the folder.
The final results are stored in a query named “Append from Folder” located in the “Other Queries”
group.
The prerequisite transformations are stored in the “Helper Queries” group.
If you need to transform the files before the append operation, the alterations to the query need to be
performed in the Helper Queries – Transform Sample File query.
Typically, the transformations can occur before or after the append operation and the results will be
identical. Performance increases can be achieved when performing certain tasks before the append
step.
A Very Helpful Step in the Query
During the final query for the append operation, there is a step that filters out hidden files.
This is an automatic behavior that prevents open Excel files from being incorporated into the append
operation. This would otherwise result in the same data being loaded twice into the output.
NOTE: If you are running an older version of Power Query that does not possess this feature, it will be
necessary to manually add a step that filters out files whose names begin with a “~” (tilde) character.
• Standardize the casing of names and/or extensions (i.e. standardize .CSV, .csv, .Csv, etc.)
• Only include files that equal a certain extension (ex: .xlsx)
• Only include files with a modification date before or after a certain day.
• Remove null rows
Copyright © 2020 Leila Gharani 45 | P a g e
Power Query Solutions to Appending Data Challenges
3. Create a new query by selecting Data (tab) → Get & Transform (group) → Get Data → From File
→ From SharePoint Folder.
4. Paste the URL copied from Step 2 into the Site URL field of the SharePoint Folder dialog box.
NOTE: If this is your initial connection to SharePoint, you will be required to supply your credentials to
gain access to the site and its content.
5. In the list of files from the SharePoint drive, click Transform Data.
6. Scroll to the column labeled “Folder Path” and filter for the desired subfolder.
NOTE: If you provide more of the path in the filter argument, the more focused (i.e. restrictive) the file
retrieval will be.
7. Rename the new query something like “SharePointDrive”. Remember this name; it will be
referenced towards the end of this solution.
8. Return to the original query that contains the file combining and transformation and select the
Source step.
9. Replace the entry in the Formula Bar with the name of the previously created query (Step 7).
10. Perform the same operation on the “Sample File” query as was performed on Step 9.
11. We do not want to load the results of the “SharePointDrive” query to an Excel table, so we will
perform a Close & Load operation as a “Connection Only” query.
NOTE: If you select multiple sheets using the “Select multiple items” and the checkboxes, Power Query
will create a separate query for each selected sheet.
NOTE: Consider filtering out sheets/tables you do not require in your analysis.
This works well until the first sheet in the workbook changes.
This will cause a following step to fail. When we rename the first column to “Company”, Power Query is
looking for a column named “Bere Kleid Data”. Since the new first sheet is named “Urban Right Data”,
the rename step fails.
Instead of looking for the column by its name, we can look for a column by its column position.
We will modify the reference to “Bere Kleid Data” with the following “M” code.
List.First(Table.ColumnNames(#”Promoted Headers”))
The tables all have the same columns (number and headings).
Issues / Considerations
We have an issue where some columns have empty cells on certain rows. The implication is that if we
are selling different items to the same Customer or on the same Sales Document, the repetitive
information will not be repeated.
This is something that will need to be taken care of when processing the query.
Creating a Query from the Ground Up
Because we don’t want to create a separate query for each sheet/table, and we don’t want to be
burdened with updates every month, we will solve this with a bit of M code.
1. Select Data (tab) → Get & Transform (group) → Get Data → From Other Sources → Blank
Query.
2. In the Formula Bar, type the following (watch your capitalization):
= Excel.CurrentWorkbook()
52 | P a g e Copyright © 2020 Leila Gharani
Power Query Solutions to Appending Data Challenges
We are presented with a table that lists all detected tables and their respective names.
NOTE: Because we are working with Excel Tables, we do not retain the heading rows from the
subsequent tables. Duplicate headings are removed automatically.
6. Filter the rows containing the words “Grand Total” from the Sales Document column.
7. Replace the nulls by selecting the Sales Document and Customer Name columns. Select
Transform (tab) → Any Column (group) → Fill → Fill Down.
8. Select all columns and assign proper data types; Transform (tab) → Any Column (group) →
Detect Data Type. Adjust as necessary (ex: Document Date to Date.)
Without changing ANYTHING, we refresh the query and observe what happens to our row count.
This means that each time the query refreshes, it will read EVERY table in the workbook, including the
output table. We refer to this as “the snake eating its tail.”
We can see from the Source step in the query that the query output table is seen as a viable input table.
To exclude the output sheet from the Refresh operation, we can insert a step in the query after the
Source step that filters the table to only include tables whose table names begin with “Data_”.Better
Collaboration – File Path as Parameter
When creating queries, it is not uncommon for our Source step to point to a local drive/folder/filename
for the data.
The problem occurs when the file(s) is sent to another user who does not maintain the same
drive/folder/file environment. We will be in constant need of updating the Source step in the query.
We can eliminate the need for Source updates by using a parameter.
Parameters can be implemented as:
• Excel Parameters
1. In Power Query, select Home (tab) → Parameters (group) → Manage Parameters → New
Parameter.
• Description
• Suggested Values
• Current Value
2. Replace the hard-coded portion of the patch (ex: C:\PQ\21) with the name of the parameter
followed by a concatenate symbol (&).
1. Create an Excel Table (or Named Range) with the heading FilePath and a single row that contains
the file’s full path. (you can use other heading names)
2. Name the table “FileExcelPath”. (you can use other table names)
3. Select a cell in the table and click Data (tab) → Get & Transform (group) → From Table/Range.
2. Replace the hard-coded portion of the patch (ex: C:\PQ\21) with the name of the parameter
followed by a concatenate symbol (&).
Power Query does not like the fact that two connections have been made within a single query; in other
words, the Path (99% of it, anyway) is coming from one query while the filename is coming from another
query.
There are two ways to deal with this issue:
b. In the Current Workbook → Privacy section and select the “Ignore the Privacy Levels and
potentially improve performance” option.
2. Resolve the path determination in the same query as the filename pointer.
a. Combine the steps in the FileExcelPath query by updating the Source step as follows:
= Excel.CurrentWorkbook(){[Name="FileExcelPath"]}[Content]{0}[File Path]
4. Highlight and copy the entire Source step from the FileExcelPath query.
a. Insert a step before the Source step by selecting Home (tab) → Query (group) →
Advanced Editor and add a blank row below the “let” statement (row 2).
b. Paste the copied data from Step 4 into the newly inserted blank row.
c. Add the label “FileExcelPath” before the equals sign of the new row.
NOTE: The below tip assumes that the report file and the source files are stored in the same folder.
You can use the Excel CELL function to generate the path/filename/sheetname of the current workbook.
=CELL(“filename”)
C:\PQ\S21\[Report.xlsx]Index
We need to extract everything to the left of the final backslash, the one just before the open square
bracket. Assuming the CELL function in cell A1, we can use additional functions to extract the needed
path.
If you wish to obtain this information in a single formula, we can combine the two formulas into one like
the following:
1. Open a blank workbook and select Data (tab) → Get & Transform (group) → Get Data → From
File → From Folder.
3. Filter the Name column, if necessary (ex: all files that contain “_Data”).
4. Filter the Extension column for specific file type (if necessary (ex: .xlsx files)
7. From the Combine Binaries dialog box, select the Data tab and click OK.
8. In the Queries panel (left), select the Transform Sample File query.
9. Delete the Promote Headers step from the Queries step list.
At this point, if the columns headings in the source files do not match in casing, nulls will result in
various locations throughout the table. To standardize the casing, we will create a custom step in the
query.
13. Select the last step in the query and click the button to the left of the Formula Bar.
NOTE: The underscore used in the Text.Proper function is a technique for referring to the entire column
as opposed to a specific column like [Company].
The headings may likely have leading and/or trailing spaces. These need to be stripped out.
15. Modify the formula from Step 14 to include the Text.Trim function before the Text.Proper
function.
17. Using the “Sales Document” column, filter out the null entries.
If you return to the final query output named “AllData”, we see that all of the data has been appended
properly based on consistent headings.
HOW TO APPEND ALL SHEETS WHEN DATA STARTS FROM DIFFERENT ROWS
It’s not uncommon for tables to start on different rows between different files.
Objectives
• Combine (append) data from separate files into a table in a new Excel file.
• Use a column heading to act as an identifier for the start of the table(s).
In this example, we will use the column heading labeled “Sales Document” as our heading row
identifier.
1. Open a blank workbook and select Data (tab) → Get & Transform (group) → Get Data → From
File → From Workbook.
3. In the Navigator dialog box, select the parent “folder” that appears above the list of sheet
names.
We need to determine the row number of each sheet where the term “Sales Document” appears in
“Column 1”
We are presented with a row result for each table where the text “Sales Document” is first encountered.
NOTE: Don’t forget that Power Query starts counting at 0 (zero). Therefore, 0 would be “row 1”.
8. To use the discovered row number information, click the gear icon next to the newly inserted
step and adjust the formula as follows:
Each table will now begin its data starting on the first row.
We can improve this by promoting the first row in each table to a header row.
9. Click the gear icon next to the newly updated step and adjust the formula as follows>
10. Select the Name column and the last column (our custom results) and remove the other
columns.
12. Filter the null values from the “Sales Document” column.
APPEND SHEETS AND GET FIELD NAME FROM A CELL OUTSIDE THE TABLE
Objectives:
Issues:
• The company’s name is located in a cell directly above the table’s header row in Column B to the
right of the word “Company”.
Solution Steps:
1. Open a blank workbook and select Data (tab) → Get & Transform (group) → Get Data → From
File → From Workbook.
3. In the Navigator dialog box, select the parent “folder” that appears above the list of sheet
names.
Create a Custom Column that locates the row number holding the word “Company”.
= List.PositionOf([Data][Column1], “Company”)
8. To use the discovered row number information, click the gear icon next to the newly inserted
step and adjust the formula as follows:
Determine the company name and place it in a separate column by way of a Conditional Column.
12. Add the following parameters to the Add Conditional Column dialog box.
13. Select the newly inserted custom column and fill the contents down the column (Fill Down).
17. Remove the redundant, embedded header rows from the data by filtering the “Sales Document”
column and deselect “Sales Document”.
Because the original header label for the company column is hardcoded to the name of the first sheet’s
company in the workbook, we need to include logic to detect the name of the first sheet’s company so
this will be dynamic and “future-proof”.
We can solve this problem with either “M” code or re-work our steps from the interface controls.
19. Select the “Filtered Rows1” step and insert a new step below this position.
= List.Last(Table.ColumnNames(#”Filtered Rows1”) )
21. Highlight the above formula in the Formula Bar and copy it to memory.
23. Select the “Renamed Columns” step and modify the instruction in the Formula Bar as follows
(replace the static reference to “Bere Kleid” with the copied formula):
20. Update the “Filtered Rows” step to include an additional filter parameter.
Data Design:
• The sales information is stored in tables where each table is dedicated to a separate company.
• The table is set up in a cross-tabular structure; Customer Name and Article Description for rows
and Months for columns.
• The Company Name is located above the table in cell B1 and within the sheet name.
• The sheets that contain needed data have their sheet names begin with “Data_”.
Solution Steps:
1. Open a blank workbook and select Data (tab) → Get & Transform (group) → Get Data → From
File → From Workbook.
3. In the Navigator dialog box, select the parent “folder” that appears above the list of sheet
names.
6. Filter the Name column to only include items that begin with “Data_”.
Create a function that will process the nested tables (Data column) before the append operation.
9. Extract the first table by clicking the Table hyperlink in the Data column.
10. Remove the automatically applied “Changed Type” and “Promoted Headers” steps.
14. Remove the nulls by filling down the Customer Name column.
15. Filter out the total rows (Does not end with… “Total”).
16. Unpivot the table by selecting the Customer Name and Article Description columns and unpivot
the other columns.
22. Add the following instruction above the let statement at the top of the query:
Use the function to process the nested tables in the original query.
25. Select the Name and TransformSample columns and remove all other columns.
27. Select the Name column and Extract → Text After Delimiter using “Data_” as the delimiter.
30. Close & Load the results to a table in the new workbook.
2. In the Create Table dialog box, verify the proper range selection and ensure the “My table has
headers” option is selected.
Step 2: Create the New Column Using the Column From Example Feature
4. Select Add Column (tab) → General (group) → Column From Example → From Selection.
5. In the test column (Custom), provide some examples of where particular salaries would fall in
your naming scheme. You may need to define 1 or 2 examples in each bucket range for Power
Query to figure out your ranges.
In the logic area, we can see sample logic that Power Query is attempting to apply based on our
suggestions.
= Table.AddColumn(#"Changed Type", "Custom", each if [Salary] >= 145000 then "High" else if
[Salary] >= 58300 then "Moderate" else if [Salary] >= 31200 then "Average" else "Low", type text)
The logic is good, but the thresholds for each bucket is incorrect.
Step 4: Refining the Logic of Columns From Example
NOTE: We could make the following changes directly in the Formula Bar, but you may feel more
comfortable using a user-friendly dialog box to relieve yourself of any risk of typographic, logic, or syntax
errors.
6. To adjust the thresholds of each bucket from Step 5, click the gear icon to open the Added
Conditional Column step.
7. In the Add Conditional Column dialog box, adjust the Values fields to reflect the desired bucket
ranges.
At this point, we can Close & Load this to a new table, a Pivot Table/Chart, or into our Data Model
7. Click the icon to the right of Path to copy the link address to memory.
8. To access the SharePoint file, select Data (tab) → Get & Transform (tab) → Get Data → From
Web and paste the copied link into the URL field and click OK.
NOTE: Because the source pointer is not pointing to your local computer, if you share this file and a user
attempts a refresh of the data, they will be required to enter their SharePoint or OneDrive for Business
credentials
9. In the Navigator dialog box, select the requisite table from the list of sources (left).
10. Name the new query something like “SalesSharePoint”. Remember this name; it will be
referenced towards the end of this solution.
11. Remove all of the steps except for the Source step.
12. We’re not going to perform any of the original transformations, so select Load → Load to…
This creates a “placeholder” query where we can extract the needed pointer for the new file location.
15. Open the original query that pointed to the local file.
17. Erase everything in the Formula Bar that pointed to the original file location/name/range.
18. In the Formula Bar, type the following formula (a reference to another query):
= SalesSharePoint
• Phone numbers are not always on the same row as the first Customer ID instance.
• Customer names are not always on the same row as the first Customer ID instance.
• Create a table with one row per Customer ID with the associated information.
Solution Steps:
Convert and Load the Data
1. Select cells A4 through C17 and convert the table to a proper Excel Table (CTRL-T).
13. For “future-proofing” purposes, select both columns and Remove Duplicates.
78 | P a g e Copyright © 2020 Leila Gharani
Power Query Solutions to Common Data Analysis Problems
14. Select Home (tab) → Combine (group) → Merge Queries → Merge Queries as New.
15. In the Merge dialog box, set the parameters shown in the following illustration.
16. Expand the nested tables in the Phone column to extract the Phone field.
• Ensure the App names are the updated App names (from the “After” column).
Solution Steps:
Load Data from File
1. Select Data (tab) → Get & Transform (group) → Get Data → From File → From Workbook.
6. Select the table of Before/After names (B4 through C10) and convert the table to a proper Excel
Table (CTRL-T).
10. Select Data (tab) → Get & Transform (group) → Get Data → Combine Queries → Merge.
11. In the Merge dialog box, set the parameters shown in the following illustration.
12. Expand the nested tables in the TransformTable column to extract the After field.
14. Set the data type for the New App Name column to Text.
• Create a new column in Power Query that calculates the difference between the current day
price and the previous day price.
Solution Steps:
Load Data from Text File
1. Select Data (tab) → Get & Transform (group) → Get Data → From File → From Text/CSV.
NOTE: If the Dates and Prices come in as errors, it’s likely due to a regional settings issue. Consider
adding steps to use Locale → Date → English (United States) and Locale → Decimal Number→ English
(United States) respectively.
4. Add an Index column by selecting Add Column (tab) → General (group) → Index Column
(starting at 0).
5. Add a second Index column by selecting Add Column (tab) → General (group) → Index Column
(starting at 1).
7. In the Merge dialog box, set the parameters shown in the following illustration.
8. Expand the column of nested tables to include the Gold Price Close column.
9. The table has lost its original sort order, so restore the sort order by sorting the Index column in
Ascending order.
10. Create the Difference column by selecting the Gold Price Close and the Gold Price Close.1
columns and clicking Add Column (tab) → From Number (group) → Standard → Subtract.
• We have an Excel Table named “TBonus” with bonus amounts based on salary thresholds.
• Create a new table that contains the employee names, their bonus rate, and the bonus amount.
Solution Steps:
Load Data from Tables
1. Select a cell in the TBonus table and load the table into Power Query as a Table.
3. Select Data (tab) → Get & Transform (group) → Get Data → From File → From Workbook.
5. Select the table named TSalary and click Load to… → Connection Only.
7. Select Data (tab) → Get & Transform (group) → Get Data → Combine Queries → Merge.
8. In the Merge dialog box, set the parameters shown in the following illustration.
11. Create a Conditional Column named “Salary Help” that detects a null in the Salary.1 column and
repeats the value in the Salary column. If a salary exists in the Salary.1 column, repeat that same
value.
12. Set the data type for Salary Help to Whole Number.
13. Add an Index column by selecting Add Column (tab) → General (group) → Index Column
(starting at 0).
14. Sort the table by the Salary Help column in Ascending order.
17. Filter out any Bonus rows that equal to zero (filter “does not equal 0”).
19. Create the Bonus Amount column by selecting the Salary and the Bonus columns and clicking
Add Column (tab) → From Number (group) → Standard → Multiply.
21. Change the data type for the Bonus Amount column to Currency.
• We have an Excel Table named “TEmployee” that consists of names, salaries, departments, and
positions on a sheet named Source.
• We have an Excel Table named “TableThreshold” that consists of departments and thresholds on
a sheet named “Report”.
• Create a report of employees whose salary is above the threshold for their respective
departments.
• Sort the results in ascending order by the department and then descending order by the salary.
• The report needs to be fully dynamic to allow for additions to the Source as well as changes to
the thresholds.
Solution Steps:
Load Data from Tables
1. Select a cell in the TableThreshold table and load the table into Power Query as a Table.
3. Select a cell in the TEmployee table and load the table into Power Query as a Table.
6. In the Merge dialog box, set the parameters shown in the following illustration.
7. Expand the nested tables in the TableThreshold column to extract all fields
8. Filter out any rows containing nulls from the TableThreshold.Threshold column.
9. Create a Conditional Column named “Include” with the parameters shown in the following
illustration that flags any Salary entries that are Greater than or Equal to the
TableThreshold.Threshold column with a “YES”. Otherwise, display a null.
10. Filter the Include column to only retain rows with the “YES” flag.
12. Sort the results in Ascending order by Department and then Descending order by Salary.
• We suspect that someone has made alterations to the company salary spreadsheet.
• We also suspect that rows may have been deleted from the spreadsheet.
• We have a backup copy of the salary spreadsheet that was made before the suspected
alterations.
• Import the current file (SalaryCurrent.xlsx) and the backup file (SalaryBackup.xlsx) into Power
Query.
• Determine which were deleted (if any) and which salaries were altered (if any).
Solution Steps:
Importing the data files
1. Open a blank workbook and select Data (tab) → Get & Transform (group) → Get Data → From
File → From Workbook.
5. Remove the top 3 rows by selecting Home (tab) → Reduce Rows (group) → Remove Rows →
Remove Top Rows.
10. Select each query and note the number of listed rows (bottom left corner).
12. Perform a Merge operation on the two tables with the parameters shown in the following
illustration.
14. Expand the nested tables from the SalaryBackup column using the following selections:
• First Name
• Last Name
• Position
• Salary
• Use original column name as prefix
NOTE: The last option will help us keep track of which file the columns originated.
16. Set the Conditional Column with the parameters shown in the following illustration.
17. Filter the Include column to only retain entries flagged “YES”.
We now can easily see which rows were removed from the worksheet (yellow) and which salaries were
changed (purple).
• We have an Excel Table named “TSales” which contains a list of dates, sales manager names,
product names, and sales values (over 2200 rows).
• We need to create a table that displays the Top 5 sales entries for each day.
Solution Steps:
Load the Date to a Query
5. Group the table by Date using the settings in the following illustration.
We will use a Custom Column function to create a list of index numbers (starting at 1) within each
nested table of the Custom column.
6. Insert a Custom Column and set the parameters shown in the following illustration.
=Table.AddIndexColumn([All], "Rank", 1)
8. Expand the column of nested tables to include all fields except Date.
9. Filter the table to only keep Ranks that are Less than or equal to 5.
10. Reorder the columns in Date – Rank – Sales Manager – Product Name – Sales order.
12. Sort the table in Ascending order by Date, then Ascending order by Rank.
• Each time a new customer name is encountered, the unique ID is to be incremented by a value of
10 (ex: 10, 20, 30, 40, etc.)
Solution Steps:
1. Select a cell in the list of customer names and load the data into Power Query as a Range (no
header row).
4. In the Group By dialog box, set the parameters shown in the following illustration.
5. Add a custom index column by selecting Add Column (tab) → General (group) → Index Column
→ Custom…
6. In the Add Index Column dialog box, set the parameters to start at 10 and increment in intervals
of 10.
7. Expand the column of nested tables named Column1 to include all fields.
• Customer names are only listed for the first article description.
• The sales are aggregated at the monthly level displayed as month names.
• Each month has a scenario for Actual along with a value for Budget.
• The headings for the scenarios “Actual” and “Budget” are stored in merged cells that traverse all
three displayed months. This results in a double-row header.
• The report needs to accommodate expansion when new months are added to the data.
1. Select a cell in the data, press CTRL-A to select the entire data range.
2. Upgrade the table to a proper Excel Table by pressing CTRL-T and do not define headers in the
table.
3. Return the table to its original look by clearing the color formatting and turning off the Header
Row option.
7. Merge the customer names (Column1) and the article descriptions (Column2) using a semicolon
as the delimiter into a new column named “Merged”.
11. Select the column of scenarios (;) and unpivot the other columns by clicking Transform (tab) →
Any Column (group) → Unpivot Columns → Unpivot Other Columns.
12. Select the Attribute column and split the column by delimiter. The delimiter is a semicolon that
is split at each occurrence.
14. Filter the Customer column to keep rows that do not end with “Total”.
16. Reorder the columns as Customer – Article – Scenario – Date – Sales Value.
17. Filter out the null values from the Sales Value column.
Summary of the steps to unpivot data with multiple row and column headers:
1. Identify anchor columns – these are the columns you want to keep
2. Fill down the anchor columns in case there are gaps
3. Merge anchor columns to create one column
4. Transpose the Table
5. Fill down the columns that have gaps
6. Promote first row to header
7. Select the new anchor columns and Unpivot Other Columns
8. Split the attribute column by delimiter
9. Continue with anything else that needs cleaning – for example filtering out total values
• We are devoid of any “anchor” columns. Power Query has no way to identify which rows belong
to which record.
• We need to assign values where each related row receives the same number.
Solution Steps:
Add a custom index column by selecting Add Column (tab) → General (group) → Index Column
(start with 0).
4. Flag the start of each group using a Conditional Column, setting the parameters as shown in the
following illustration.
7. Select the Field Name column and click Transform (tab) → Any Column (group) → Pivot
Column. Set the Values Column to Value and the aggregation method to Don’t Aggregate.
More Resources
Power Query Learn More
MORE RESOURCES
In addition to this Solution book, we’ve also put together a Power Query Reference book. In the
Reference book you’ll get more details about how you can use each Power Query feature. This book is
also available as a resource inside your online Power Query course on XelPlus.com.
Let's talk about more learning.
On my site, you’ll find weekly Excel articles on different aspects of Excel: XelPlus.com/tutorials/
There are a lot of great Excel websites out there. For more Excel information, check out my list of
resources here: XelPlus.com/resources
The more you know of Excel the easier it gets for you to implement complex projects.
If you're interested to become more advanced and create great visualizations or effective dashboards,
check out my other courses: https://www.xelplus.com/courses.
I also share weekly tutorial videos on my YouTube channel so make sure you check it out if you're
interested to learn more. Just type in Leila Gharani on YouTube.
I wish you the best of luck on your journey and hope to see you in my other courses.