Ski Resorts Cleaning

This page describes the cleaning and aggregation process for the main Ski Resorts dataset.

The cleaning and aggregation module Python script can be found here (Cleaning - Ski Resorts).

Main Ski Resort Data

Snippet of Initial Main Ski Resort Data

Null Values of Initial Main Ski Resort Data

Google Geolocation API

Using a function created in Source - Google Functions, the first step was to use the scraped address in conjunction with the Google Geolocation API to find coordinates for each ski resort.

Epic and Ikon Data

Snippet of Epic Pass Data

Null Values of Epic Pass Data

Snippet of Ikon Pass Data

Null Values of Ikon Pass Data

Matching Resort Names

A stated requirement within the sources section was to find consistent naming within our resort datasets. This played a driving role in the creation of the function match_resorts(). Due to inconsistancies within the resort names across the three sources, after an initial merge, not all resorts were a match. This function returns a list of the closest responses and lets a user decide which resort is the actual match. After this was ran for both Epic and Ikon, and then the three sources were merged together resulting in a final dataset with the Pass type for each resort.

Proper Address Components

When the main ski resort data was initially scraped, there was an attempt to extract columns of general location, such as region, state, and city. This proved to be inconsistent and overall unsuccessful. To account for this, address components such as state (United States) and province/territory (Canada), as well as city were extracted via a backwards application of the Google Geolocation API. This actually included a step into using the Google Places API as well. When state/province/territory and city were not available during the first iteration of obtaining proper address components, nearby business locations were iterated through until the required components were found.

Snippets and null values for the proper address and address unpacking are found below.

Resort Proper and Address Unpacked

Snippet of Resort Proper

Null Values of Resort Proper

Snippet of Address Unpacked

Null Values of Address Unpacked

Address Unpacked Explanation

By referencing the results above, "long_name" and "locality_type_0" were results of interest. Values of interest contained in "locality_type_0" are "administrative_area_level_1" and "locality", which are equivalent to "state/province/territory" and "city", respectively.

Cleaning Resort Name

Again back to the consistent resort naming, the function clean_text_v1() was created to aid in this endeavor. The main drivers which prompted this were non-uniform alphabetical characters, mainly due to French areas in Canada. Additionally, some resorts had unnecessary details in parentheses and extra spaces. Text was normalized.

Dropping Columns and Rows

In an effort to retain as much information as possible, columns and rows were dropped intelligently. Rows containig an "indoor resort" indicator were removed, as the scope of this project is mountainous outdoor resorts. Some columns with an unrecoverable amount of missing data or were unnecessary were removed as well:

• Region: only has North America
• Country: lowercase country has longname
• Locale 1: administrative_area_level_1 & locality better contain a better summary
• Locale 2: administrative_area_level_1 & locality better contain a better summary
• Link: only required for scraping purposes
• Address: messy and administrative_area_level_1 & locality better contain a better summary
• Snow reliability: over 84% missing values
• Slope preparation: over 84% missing values
• Access, on-site parking: over 84% missing values
• Snow reliability: over 84% missing values
• Orientation (trail map, information boards, sign-postings)
• Cleanliness and hygiene: over 84% missing values
• Environmentally friendly ski operation: over 84% missing values
• Friendliness of staff: over 84% missing values
• Mountain restaurants, ski huts, gastronomy: over 84% missing values
• Après-ski: over 84% missing values
• Accommodation offering directly at the slopes and lifts: over 84% missing values
• Families and children: over 84% missing values
• Beginners: over 84% missing values
• Advanced skiers, freeriders: over 84% missing values
• Snow parks: over 84% missing values
• Cross-country skiing and trails: over 84% missing values
• Epic Area: administrative_area_level_1 & locality better contain a better summary
• Ikon Area: administrative_area_level_1 & locality better contain a better summary

Filling Null Values

Columns still containig null values were:

• Price: 8.7% missing
• Pass: 82.1% missing

Solutions for these missing values were found:

• Price: since resort rating had no null values, the average price of resorts with a nearby rating (tolerance of 0.1) was used.
• Pass: although 82.1% of missing data is usually an indicator to drop a column, this simply meant the resort wasn't under Epic or Ikon, and was given the value of "Other".

Resort Proper and Address Unpacked

One of the final steps was to add another general location indicator, region. The United States and Canada each have major regions which seperate mainly geographical features and may be decent paritions for analysis.

Snippets of the raw scraped data and null values follow:

Snippet of US Regions

Null Values of US Regions

Snippet of Canada Regions

Null Values of Canada Regions

The Final Resort Data

After applying cleaning, adjustments, and merging of the components from major regions, the final cleaned resort dataset was formed. Additionally, more concise column naming was applied.

Snippet of Final Resort Data

Null Values of Final Resort Data