Consuming and Storing Data from a REST Service with ASP.NET Razor

This tutorial will illustrate how to consume and store data from a web service using WebMatrix. WebMatrix is a comprehensive, lightweight web development tool that includes everything you need for building websites. This tutorial assumes that you are already familiar with some of the basic concepts involved in working with WebMatrix. You will be working with ASP.NET Web Pages with Razor syntax to write server code. The new Razor syntax is based on a technology from Microsoft called ASP.NET, which in turn is based on the Microsoft .NET Framework. Razor gives you all the power of ASP.NET, but using a simplified syntax that's easier to learn if you're a beginner and makes you more productive if you're an expert. You will also be working with basic SQL statements and the database features integrated with WebMatrix. If you are unfamiliar with any of these features take a quick look at the following tutorials before continuing.

http://www.microsoft.com/web/post/web-development-101-using-webmatrix

http://www.asp.net/webmatrix/tutorials/2-introduction-to-asp-net-web-programming-using-the-razor-syntax

http://www.microsoft.com/web/post/web-development-101-part-5-using-data

Getting Started

Let’s begin with a brief review of how Razor code is stored and accessed in a WebMatrix site. One of the more powerful features of the Razor syntax is the ability to create code blocks that execute on the server. Code blocks let you wrap up reusable code in a single location and then provide a simple statement for accessing the code in your client content. To create a reusable code block, you simply add cshtml files to the App_Code folder in the root of a WebMatrix site. In some of the site templates offered by WebMatrix, the App_Code folder is not always included. If you are working on a site that does not have the App_Code folder available, you can create it as you would any other folder. Any cshtml file that is added to this location will be accessible by any other page within the site using the following syntax:

FileName.FunctionName([DataType arg], [DataType arg]…);

When adding Razor code blocks to files that are stored in the App_Code folder, Razor requires the code to be wrapped in a code block statement using the following syntax:

@functions{

}

The functions keyword allows any syntactically valid code, including: variables, methods and objects to be executed on the server. Methods and variables that are declared within the functions code and are marked with the public and static keywords will be accessible using the Razor ‘@’ character on any page of the site. So if you created a file called “MyFunctions.cshtml” in the App_Code folder and added the following function in a code block:

@functions{
      public static string HelloWorld()
      {
            return “Hello World!”;
      }
}

You would be able to add the following inline expression to any page in a site and have the words “Hello World” added the web page output:

@MyFunctions.HelloWorld();

Now that you have a better understanding of how we will be working with Razor code, let’s move on and look at how we can use reusable code blocks to help a WebMatrix site communicate with a remote web service.

Choosing a Weather Service

For this tutorial we will be working with the weather forecast services provided by the National Oceanic and Atmospheric Administration (NOAA). While there are numerous weather forecast services available, the detail and amount of data that can be retrieved from NOAA will provide us with a nice dataset to work from as well as allow us to explore some of the features of WebMatrix.

The NOAA publishes its weather data using two different web service standards: Simple Object Access Protocol (SOAP) and Representational State Transfer (REST). Although both standards operate by sending data over HTTP, the SOAP standard requires an application to application specific overlay of HTTP that becomes more complex as a service grows in size. REST, on the hand, attempts to retain full compatibility with HTTP regardless of complexity by communicating using the same context as a web page. The combination of simplicity and compatibility make the RESTful approach a good fit for WebMatrix sites, therefore we will continue by taking a closer look at how the NOAA REST service works.

NOAA weather forecast data is requested by passing different groupings of query parameters to predetermined URL’s. The following tables provide a brief overview of the parameters that will be used in this tutorial.

Parameter Name

Example

Description

listLatLon

47.6801,-122.121

The latitude and longitude pair for the area of the grid to retrieve data.

format

12 hourly

24 hourly

Determines how data is organized in the markup. Select “12 hourly” to summarize data in two twelve hour periods (6:00 AM to 6:00 PM and 6:00 PM to 6:00 AM)

or “24 hourly” to summarize data into a single time period.

numDays

1

The number of days ahead of the request date to return data.

The parameters listed above will be passed to the following URL as a query string:

http://www.weather.gov/forecasts/xml/sample_products/browser_interface/ndfdBrowserClientByDay.php

Using the “numDays” parameter, you can request forecast data for one all the way up to ten days into the future. However, the more days requested the larger the dataset that is returned becomes. So to keep things simple, we will only be focusing on weather information for the current day. The URL for this request would then look like the following:

http://www.weather.gov/forecasts/xml/sample_products/browser_interface/ndfdBrowserClientByDay.php?listLatLon=47.6801,-122.121&format=24+hourly&numDays=1

The “listLatLon” parameter is used to determine the geographical location of the desired weather forecast. Since working with longitude and latitude numbers can be a little cumbersome, we will also be using a second service that the NOAA provides for converting zip codes.

Parameter Name

Example

Description

listZipCodeList

98052

The zip code of the area for which you want longitude and latitude grid points.

The zip code service uses a URL that is slightly different from the weather service:

http://www.weather.gov/forecasts/xml/sample_products/browser_interface/ndfdXMLclient.php?listZipCodeList=98052

Notice that instead of calling the “ndfdBrowserClientByDay.php” page, we make the request from “ndfdXMLclient.php”, and then add the query string using the same format. The service responds with a geographical coordinate pair that can then be added as a parameter value to the weather service URL.

Now that we have a functional understanding of how the NOAA REST services are accessed, let’s look at how we can access the data using Razor.

Communicating with a REST Service

NOAA services are based on a form of XML called Digital Weather Markup Language (DWML). The following DWML is what you would see if you called the zip code service from a browser.

<dwml version="1.0" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:noNamespaceSchemaLocation="http://www.nws.noaa.gov/mdl/survey/pgb_survey/dev/DWMLgen/schema/DWML.xsd">
  <latLonList>47.6801,-122.121</latLonList> 
  </dwml>

Since we want to use the data contained within the DWML in a WebMatrix site we will need a way to call the service using Razor. If you remember from the introduction, Razor is built on top of the .NET Framework. This means we have access to a comprehensive programming framework that will make working with REST services and DWML pretty easy. The “System.Web” namespace of the .NET Framework contains a class called WebClient that provides functionality for sending and receiving data from any website using a valid URL. That means we can use WebClient to call the NOAA service URL’s we looked at in the last section to retrieve DWML.

The first thing you will want to do is create a new Razor file in the App_Code directory. If you haven’t already and would like to follow along, go ahead and create a new WebMatrix site using an empty site template. Since we are working with weather forecast data, let’s call the file “Weather.cshtml”. We also need a content page that we can use to render the output from the weather service to a browser. So let’s create one more file at the site root and call it “Default.cshtml”.

As we discussed in the last section, before we can request weather data we need to know the longitude and latitude for a given area. So we will start by creating a method that takes a zip code as a parameter and uses the zip code service to return a geographical coordinate pair. Add the following code at the beginning of the Weather.cshtml file:

@functions{
    private static string WeatherService = 
"http://www.weather.gov/forecasts/xml/sample_products/browser_interface/ndfdBrowserClientByDay.php?listLatLon={0}&
format=24+hourly&numDays=1";
    private static string ZipcodeService = 
"http://www.weather.gov/forecasts/xml/sample_products/browser_interface/ndfdXMLclient.php?listZipCodeList={0}";

    public static string GetCoordinates(string zipcode){
        string serviceURL = string.Format(ZipcodeService, zipcode);
        string dwml = string.Empty;
        
        System.Net.WebClient webClient = new System.Net.WebClient();
        dwml = webClient.DownloadString(serviceURL);
        return dwml;
    }
}

The two private strings, “WeatherService and ZipcodeService” will be used to hold the URL for each service. The URL parameters that need to change contain placeholders that can be replaced whenever desired using the formatting features of the String object. Notice that to use WebClient all we needed to do was call the DownloadString method and pass it the desired URL.

To run this code, simply add the following inline Razor code anywhere within the body of the Default.cshtml file.

@Weather.GetCoordinates("98052");

When you run the site you should see the exact same content that would be displayed if you had opened the URL directly in a browser. This is because WebClient is performing the same actions that a browser does; you pass it a URL and it returns the data. The only difference is that we are first handling the data in Razor server code and then writing to the page.

Now that we have a function for requesting the geographical coordinates of a zip code, let’s create another method that will use those coordinates to request weather forecast data. Just add the following code on a new line after the closing brace of the GetCoordinates function.

public static string GetWeather(string coordinates){
        string serviceURL = string.Format(WeatherService, coordinates);
        string dwml = string.Empty;
        
        System.Net.WebClient webClient = new System.Net.WebClient();
        dwml = webClient.DownloadString(serviceURL);
        return dwml;
    }

Once again, we can perform a quick test on this method by replacing the GetCoordinates method with GetWeather.

@Weather.GetWeather("47.6801,-122.121");

The coordinates used in the sample above are taken from the DWML data returned by the GetCoordinates method. Now the next thing we want to do is automate this process by adding some parsing logic to read and extract data from the DWML.

Since DWML is XML-Compliant, we can take advantage of the XML to LINQ parsing libraries available in the .NET Framework. So let’s revisit the GetCoordinates function and insert some code that will allow us to just return the coordinates as a string instead of the entire DWML response.

private static string GetCoordinates(string zipcode){
    string serviceURL = string.Format(ZipcodeService, zipcode);
    string dwml = string.Empty;
    
    System.Net.WebClient webClient = new System.Net.WebClient();
    dwml = webClient.DownloadString(serviceURL);
    
    System.Xml.Linq.XDocument xdoc = System.Xml.Linq.XDocument.Parse(dwml);
    var coordinates = xdoc.Descendants("latLonList").FirstOrDefault();
    if(coordinates == null)
        return string.Empty;
    else
        return coordinates.Value;
}

We know from looking at the DWML source that the coordinates that we need are a string value of the latlonlist element. Therefore, once the DWML is parsed, we can use LINQ to extract all the latlonlist elements. One thing to note is that XML to LINQ assumes that any valid XML element could be used more than once. So whenever we want to access an element we have to deal with a collection. Additionally, it is possible to retrieve data for multiple geographical locations in a single service request so there may be times where working with coordinates is desirable. However, in our example we simply need to access a single coordinate so we can safely request the first element that is found using the FirstOrDefault extension method on the collection.

In the GetWeather method we will extract a series of data points from the DWML to properly build a weather forecast. Unlike the GetCoordinates method, we will need to return more than just one value. One way to handle this is by creating a custom object that contains some simple properties that can hold different values. You can insert the following code anywhere within the @functions code block on the Weather.cshtml page.

public class Temperature
{
    public string Zip { get; set; }
    public string Latitude { get; set; }
    public string Longitude { get; set; }
    public int MaxTemp { get; set; }
    public int MinTemp { get; set; }
    public string Forecast { get; set; }
}

The Temperature class gives us a nice clean container for working with the weather forecast data of a particular zip code. So let’s revise the GetWeather method to include the use of the Temperature class.

private static Temperature GetWeather(string zipcode){
    string coordinates = GetCoordinates(zipcode);
    if(coordinates == string.Empty){
        return new Temperature();
    }
    string serviceURL = string.Format(WeatherService, coordinates);
    string dwml = string.Empty;
    
    System.Net.WebClient webClient = new System.Net.WebClient();
    dwml = webClient.DownloadString(serviceURL);
    return XmlToTemperature(dwml, coordinates, zipcode);
}

Notice that now instead of returning a string, GetWeather returns an instance of the Temperature class. The parsing required for the DWML that we receive from the weather data request is a little bit more involved, so we will break it out into a new function called XmlToTemperature. Since we will not need to provide access to this function anywhere else in the WebMatrix site, we can mark it as private instead of public. You can add the following code on a new line after the GetWeather function.

private static Temperature XmlToTemperature(string xml, string coordinates, string zipcode){
     System.Xml.Linq.XDocument xdoc = System.Xml.Linq.XDocument.Parse(xml);

    var maxTemp = from t in xdoc.Descendants("temperature").Elements("value")
                  where t.Parent.Attribute("type").Value == "maximum"
                  select t;

    var minTemp = from t in xdoc.Descendants("temperature").Elements("value")
                   where t.Parent.Attribute("type").Value == "minimum"
                   select t;

    var max = maxTemp.FirstOrDefault().Value;
    var min = minTemp.FirstOrDefault().Value;

    string forecast = xdoc.Descendants("weather-conditions").Attributes("weather-summary").FirstOrDefault().Value;
    string[] lonlat = coordinates.Split(',');
    
    Temperature temp = new Temperature();
    temp.MaxTemp = Convert.ToInt32(max);
    temp.MinTemp = Convert.ToInt32(min);
    temp.Zip = zipcode;
    temp.Latitude = lonlat[0];
    temp.Longitude = lonlat[1];
    temp.Forecast = forecast;

    return temp;
}

XmlToTemperature parses the DWML that contains the weather forecast data. We also have stored the zip code and coordinate information in each Temperature object so that it will be available if we ever need it in the future.

Now let’s run the revised code and see how the changes we made have allowed the site to gain more control over the weather forecast data that we retrieve from the service. The changes we have made will require some updates to the client code that was written earlier. Since the GetWeather method is returning a Temperature object, we will need to modify how the data is rendered to the page. We can do this by adding a code block to the top of the default.cshtml file and adding a call to the GetWeather method:

@{
    var temp = Weather.GetWeather("98052");
}

Next we’ll replace the code in the body of the file with an ordered list that contains labels for each weather data value that we want to display.

<ol>
    <li>Zip code: @temp.Zip</li>
    <li>High: @temp.MaxTemp</li>
    <li>Low: @temp.MinTemp</li>
    <li>Forecast: @temp.Forecast</li>
    <li>Longitude: @temp.Longitude</li>
    <li>Latitude: @temp.Latitude</li>
</ol>

The temp variable holds the temperature object that was returned by GetWeather, so all we have to do is call the properties contained within the object. Each property value will be written to the client so that when the page is rendered an ordered list of data is displayed.

1-1

As you can see in the screen capture, we now have a nice clean and reusable way to represent the weather forecast data in a WebMatrix site.

Caching with SQL Compact

So far in this tutorial we have successfully used Razor to interact with a weather service and parse the resulting data into a custom object that can easily be used throughout a WebMatrix site. Using a service like this provides a lot of options for building interesting web applications. However, what happens when that service slows down or is unavailable? If you want to ensure that the data you are using from an external service does not cause an interruption on your site, it might be worth implementing some form of data cache.

WebMatrix has been built with some powerful, but easy to use database tools that make storing and retrieving data using Razor clean and simple. To demonstrate this, we’ll create a database that can be used to store the Temperature objects that we create after a call to the weather service. Using the Databases Workspace within WebMatrix add a new database to the site called WeatherCache.sdf. Once the database has been created, add a table named CurrentTemperature and use the following schema to create the table columns.

Column Name

Data Type

Allow Nulls

zipcode

nvarchar

False

forecast

ntext

True

maxtemp

int

True

mintemp

int

True

latitude

ntext

True

longitude

ntext

True

timestamp

datetime

True

You may notice that the CurrentTemperature table column names match the properties of the Temperature object we created in the last section. This will help us keep a nice one to one relationship between the data in code and how it is stored in the database.

Now that we have a database configured, let’s return to the Weather.cshtml file we created earlier and add some caching logic. We want to add some code that will allow us to control when the site requests data from the service and when it pulls the data locally from the database. Add the following code after the closing brace of the GetWeather function.

public static Temperature GetForecastByZipcode(string zipcode){
    //Initialize a Temperature object.
    Temperature temp = new Temperature();
    
    //Connect to the WeatherData database and check for existing data.
    var db = Database.Open("WeatherCache");
    var currentTemp = db.QuerySingle("SELECT * FROM CurrentTemperature WHERE zipcode = @0", zipcode);
    
    //if result is null, then weather data has not been entered for the request zipcode.
    //so, add weather data.
    if(currentTemp == null){
        temp = AddWeatherToCache(zipcode);
    }else{
        //weather data exists, so now check to see if it has expired.
        DateTime timeStamp = currentTemp.timestamp.AddHours(1);
        
        if(timeStamp < DateTime.Now){
            temp = UpdateWeatherCache(zipcode);
        }else{
            //the cache has not expired, so create a update the Temperature object with cached data.
            temp = new Temperature(){
                Zip = zipcode,
                Latitude = currentTemp.latitude,
                Longitude = currentTemp.longitude,
                MaxTemp = currentTemp.maxtemp,
                MinTemp = currentTemp.mintemp,
                Forecast = currentTemp.forecast
            };
        }
    }
    return temp;
}

The GetForecastByZipcode function will be the main entry point used by content pages that want to check the weather forecast by using the cache. This function uses the zip code parameter to perform a query on the WeatherCache database. If a record does not exist, then a call to the weather service is required. If the record does exist, then the timestamp for that record is used to determine if the cache has expired. The NOAA weather service is updated about once an hour, so setting the cache expiration to one hour will ensure that this site will always have the latest data. If the cache is safe to use, then a new Temperature object is created and set with the record data.

If the currentTemp variable is null, that means the database query was unable to find a record for the requested zip code. If a record does not exist, it should be added to the database to begin the caching process. The AddWeatherToCache function takes the requested zip code and calls the GetWeather that we are using to create Temperature objects from the NOAA service. The data stored in the properties of the Temperature object is then added to the database as a new record. The time stamp for the cache is set to the current time.

private static Temperature AddWeatherToCache(string zipcode){
    Temperature temp = GetWeather(zipcode);
    if(temp.Zip == null)
        return temp;
    
    var db = Database.Open("WeatherCache");
    db.Execute("INSERT INTO CurrentTemperature VALUES (@0,@1,@2,@3,@4,@5,@6)", temp.Forecast, temp.MaxTemp, temp.MinTemp, 
DateTime.Now, temp.Latitude, temp.Longitude, zipcode);
    return temp;
}

The records that do exist, but have expired are updated using the same process as AddWeatherToCache. However, instead of performing an insert on the CurrentTemperature table, an update is used. Notice that since we already have zip code and coordinate data in the record, only the forecast, temperature and timestamp values are updated.

private static Temperature UpdateWeatherCache(string zipcode){ 
    Temperature temp = GetWeather(zipcode);
    if(temp.Zip == null)
        return temp;
    
    var db = Database.Open("WeatherCache");
    db.Execute("UPDATE CurrentTemperature SET forecast=@0, maxtemp=@1, mintemp=@2, timestamp=@3 WHERE zipcode = @4", 
temp.Forecast, temp.MaxTemp, temp.MinTemp, DateTime.Now, zipcode);
    return temp;
}

And that’s it. Now all that is left to do is update the Default.cshtml page so that temperature requests are processed through the cache instead of directly from the NOAA weather service. All we need to do is replace the call to the GetWeather function with a call to GetForecastByZipcode so that the code block now looks like the following:

@{
    var temp = Weather.GetForecastByZipcode("98052");
}

Now when the site is run it will only request weather data from the NOAA service when either a new zip code is entered or a zip code has not been checked within the last hour. You should notice a significant speed increase whenever a zip code request is coming from the cache.

Summary

In this tutorial you saw how easy it is to get up and running quickly with WebMatrix. By combining the simplicity of the Razor syntax with the depth and power of the .NET Framework you learned how to add complex features like remote service calls and data caching to an empty site with only a few lines of code.

WebMatrix was created from the ground up to be a simple, all-inclusive web development tool that is easy to use but still powerful enough to deliver the advanced features used by websites today.

Additional Resources

Download WebMatrix http://www.microsoft.com/web/webmatrix/

Learn more http://www.microsoft.com/web/category/learn

More tutorials http://www.asp.net/webmatrix

Twitter http://twitter.com/mswebplatform

Facebook http://www.facebook.com/WebPlatform

For any questions or comments about this tutorial please contact the Aeshen team at http://www.twitter.com/aeshen

You can discuss this article using the adjacent Facebook talkback.

For technical questions please visit our discussion forums, where we have a vibrant community of developers like you, as well as Microsoft engineers who are ready to answer your questions!