PHP Starter » Weather

Parse ZFP (Zone Forecast Product) Data in PHP – Option 1

Andrew — Wed, 03 Mar 2010 14:00:59 +0000

In a previous article, I covered 5 Sources of Free Weather Data for your Site, but did not provide any actual code to use the data. Since then, I covered sources #1 and #2. None of these sources had any English-formed forecasts. For that, we have to go to the Zone Forecast Product, but we don’t have the luxury of XML here. There is a healthy bit of RegEx and string manipulation, but you can do it!

Where to Get the Data

The data is available by browsing the http://www.weather.gov/data/ directory. Here, you will find a whole set of products available. To access the ZFP data, you need to browse the http://www.weather.gov/data/XXX/ZFPXXX directory where ‘XXX’ is the code for the local station from where the data is to be retrieved from. For example, if I wanted the ZFP data for KLOT (Lockport, IL), I would call up http://www.weather.gov/data/LOT/ZFPLOT.

There is an exception to this rule, however. If you look at http://www.weather.gov/data/AFC/, they show the ZFP for two areas. I think some of these are like this because one office is forecasting for to areas or something (correct me in the comments if I’m wrong). So that formula above is more of a guideline for finding the right link for your area, but I found it to be 95% accurate. (That’s still an ‘A’, right?)

Understanding the Big Picture

Click on the thumbnail for the explained section of this data. Starting at the top is the header. We don’t use this. It contains the station ID and the type of product this is – we know that already. It also contains the release time. Although that is significant, that same information is in the header of the individual forecasts as well, so I will be discarding that in this example.

The second yellow box is the header for that specific forecast zone. We will be using some data from this. The zone code is the most critical because we need to know for which zone this forecast is for. The only other piece of information in this box that I use is the Release Time, which is when the forecast was released (ironic). This block of information is repeated at the top of every forecast in the file.

Below the second yellow box is the actual forecast data, which is the meat of what we are looking for. Below that is the separator, which is means that the code block for the next forecast zone is next, and so on…

Breaking up the Forecasts

We only want to deal with one forecast zone at a time, so lets break them up. First, I need to get rid of that pesky header at the top of the file. We do that by splitting the data by the double line breaks, and removing that first paragraph from the resulting array. Then, join it back together.

/* remove the file header */

/* $data = {text from http://www.weather.gov/data/LOT/ZFPLOT} */

$data = trim($data);
$data = explode("\n\n", $data);
unset($data[0], $data[1]);
$data = implode("\n\n", $data);

Now, we can split of the zone forecasts. We simply split the text by the separator (‘$$’), and keep the forecasts in an array.

/* separate the forecasts */

/* $data = {result from previous example} */

$data = explode('$$', $data);
/* trim off any extra line breaks */
$data = array_map('trim', $data);

Perfect. Now we have an array or forecasts with their respective header block.

Parsing the Zone Codes

This is the hardest part of understanding the data. We need to know for which geographical area the forecast is for. Most zone forecasts are only for one zone, but it is possible that it may be for more than one. In the above example, the zone code is ILZ014, which is zone 14 of Illinois. Unfortunately, the zone code & purge date line can be as complicated as ILZ001>003-005-IAC045-163-051200-, which is Illinois zone 1, 2, 3, 5 as well as Iowa counties 45 and 163. For the full specs on how to understand this, see the Universal Geographic Code specifications.

Before we can do anything, we need to find the zone codes. To do that, we need to use one heck of a regular expression, found on line 8 of the sample below. This is what I used to verify that I am finding everything correctly.

Run This Example

|-)[0-9]{3})*)-/';

/* for this example, we are doing a replacement to show we found everything...
	normally, something like preg_match() would be used */
$data = preg_replace($regex, '' . "\${1}" . '-', $data);

/* do I really need to explain this? */
echo '' . $data . '';

/**
 * Why do I comment out the PHP closing tag?
 * See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
 */
/* ?> */

We’re not done with this yet. As you can see from the Universal Geographic Code reference page, they like to make shortcuts. For example, they will use INZ021-028>031 instead of INZ021-INZ028-INZ029-INZ030-INZ031. From a programming standpoint, I think we both know which is better to understand. When I put this data in a database, I need to be able to query a specific location and see if there is data for it. So, if I store it as INZ021-028>031, how do I query for zone 29? These ranges will have to be expanded.

I’m not going to go into great detail on how to do this…I will just provide a couple functions to do it. Call parse_zones() like I do at line 69, and it will take care of the rest.

Run This Example

|-)[0-9]{3})*)-/';
	
	$count = preg_match_all($regex, $data, $matches);
	$total_zones = '';
	
	foreach ($matches[0] as $field => $value)
	{
		/* since the NWS thought it was efficient to not repeat state codes, we have to reverse that */
		$state = substr($value, 0, 3);
		$zones = substr($value, 3);
		
		/* convert ranges like 014>016 to 014-015-016 */
		$zones = expand_ranges($zones);
		
		/* hack off the last dash */
		$zones = substr($zones, 0, strlen($zones) - 1);
		$zones = $state . str_replace('-', '-'.$state, $zones);
		
		$total_zones .= $zones;
	}
	
	
	$total_zones = explode('-', $total_zones);
	return $total_zones;
}

/**
 * The NWS combines multiple zones into ranges...not good for our purpose
 * All we want to do here is convert ranges like 014>016 to 014-015-016
 * See: http://www.weather.gov/emwin/winugc.htm
 */
function expand_ranges($data)
{
	$regex = '/(([0-9]{3})(>[0-9]{3}))/';
	
	$count = preg_match_all($regex, $data, $matches);
	
	foreach ($matches[0] as $field => $value)
	{
		list($start, $end) = explode('>', $value);
		
		$new_value = array();
		for ($i = $start; $i <= $end; $i++)
		{
			$new_value[] = str_pad($i, 3, '0', STR_PAD_LEFT);
		}
		
		$data = str_replace($value, implode('-', $new_value), $data);
	}
	
	return $data;
}


$zones = parse_zones("INZ021-028>031-035-036-190915-");

header('Content-type: text/plain');
var_dump($zones);

/**
 * Why do I comment out the PHP closing tag?
 * See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
 */
/* ?> */

Putting it All Together

So we know how to download the latest forecast data for a specific station, split it up, and get the zones for each one. Now it’s time to put it together and in a format where we can store or display it. In this example, I am adding some more basic PHP to form a large array with the forecast block and zones in each element. From there, you can easily place the data in a database or whatever.

Note: Note that I am using data from KIND and not KLOT in this example, because KIND has forecast blocks that cover multiple zones, showing how parse_zones() work.

Run This Example

 $value)
{
	$lines = explode("\n", $value);
	$zones = parse_zones($lines[0]);
	
	$blocks = explode("\n\n", $value);
	$forecast = $blocks[1];
	
	$data_time = parse_date_time($value);
	
	$data_form[] = array('zones' => $zones, 'date_time' => $date_time, 'forecast' => $forecast);
}


header('Content-type: text/plain');
var_dump($data_form);


/**
 * Why do I comment out the PHP closing tag?
 * See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
 */
/* ?> */

After running the example, you will see that we have an array with several numbered elements. Each element is a sub-array containing an array of zones and an element containing the forecast data.

So there you have it. With one request to the NWS website, you can gather all the ZFP data for an entire forecast area. Again, this is useful when you need to gather forecasts for the general area, not just one zone at a time. If you only need to grab one zone at a time, stay tuned because I will be writing soon on how to make use of an easier data source that grabs one zone at a time that makes for easier parsing on your part.

Making it Look Pretty

Another teaser for a future article…Once you find the page containing the data you want to work with, it looks like an undaunted task to turn that data into something presentable. Let me assure you right off the bat that it is doable with a healthy bit of string manipulation and regular expressions. Just for a bit of comparison, you should be able to turn this into this. How do we split up the days and add the cool icons? I will cover that in the next article. To be continued…

No related posts.

Merge Image Layers in PHP

Andrew — Fri, 17 Jul 2009 12:00:40 +0000

In a previous article, I showed how to Locate the Nearest Radar Station and Display Radar Images. A commenter, mike, pointed out the idea of merging the image layers using PHP instead of layering all the images separately using CSS. Here is how to do it.

Just like the previous article on finding the closest radar station, I am going to use IP geolocation to find the closest radar site. See that previous article for an explanation on how to do that.

Run This Example

fetch($url);
		file_put_contents($filename, $snoopy->results);
		$snoopy->results = '';
	}
}

/* download the radar images */
$radar_id = $radar['id'];
dl_cache("http://radar.weather.gov/Overlays/Topo/Short/{$radar_id}_Topo_Short.jpg", "maps/{$radar_id}_Topo_Short.jpg");
dl_cache("http://radar.weather.gov/RadarImg/N0R/{$radar_id}_N0R_0.gif", "maps/{$radar_id}_N0R_0.gif");
dl_cache("http://radar.weather.gov/Overlays/County/Short/{$radar_id}_County_Short.gif", "maps/{$radar_id}_County_Short.gif");
dl_cache("http://radar.weather.gov/Overlays/Highways/Short/{$radar_id}_Highway_Short.gif", "maps/{$radar_id}_Highway_Short.gif");
dl_cache("http://radar.weather.gov/Overlays/Rivers/Short/{$radar_id}_Rivers_Short.gif", "maps/{$radar_id}_Rivers_Short.gif");
dl_cache("http://radar.weather.gov/Overlays/Cities/Short/{$radar_id}_City_Short.gif", "maps/{$radar_id}_City_Short.gif");
dl_cache("http://radar.weather.gov/Legend/N0R/{$radar_id}_N0R_Legend_0.gif", "maps/{$radar_id}_N0R_Legend_0.gif");
dl_cache("http://radar.weather.gov/Warnings/Short/{$radar_id}_Warnings_0.gif", "maps/{$radar_id}_Warnings_0.gif");

/* load the radar layers into an array */
$layers = array();
$layers[] = imagecreatefromjpeg("maps/{$radar_id}_Topo_Short.jpg");
$layers[] = imagecreatefromgif("maps/{$radar_id}_N0R_0.gif");
$layers[] = imagecreatefromgif("maps/{$radar_id}_County_Short.gif");
$layers[] = imagecreatefromgif("maps/{$radar_id}_Highway_Short.gif");
$layers[] = imagecreatefromgif("maps/{$radar_id}_Rivers_Short.gif");
$layers[] = imagecreatefromgif("maps/{$radar_id}_City_Short.gif");
$layers[] = imagecreatefromgif("maps/{$radar_id}_N0R_Legend_0.gif");
$layers[] = imagecreatefromgif("maps/{$radar_id}_Warnings_0.gif");

$image = imagecreatetruecolor(600, 550);


/* merge the layers */
for ($i = 0; $i < count($layers); $i++)
{
	imagecopy($image, $layers[$i], 0, 0, 0, 0, 600, 550);
}

/* we're done! output the image... */
header('Content-type: image/jpeg');
imagejpeg($image);

/**
 * Why do I comment out the PHP closing tag?
 * See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
 */
/* ?> */

The code from line ~31 on is pretty self-explanatory. Basically, I am caching the maps locally, and then loading the images into PHP image resources. I am putting the images in a PHP array because I want to make it easy add/delete/move layers. For the application I’m using it for, it’s not likely, but your needs may differ.

The important function is imagecopy(), which is defined as:

bool imagecopy ( resource $dst_im , resource $src_im , int $dst_x , int $dst_y , int $src_x , int $src_y , int $src_w , int $src_h )

Since I am copying whole pictures directly on top of each other – all the same size – it’s very easy. You can specify image sizes and location in case you are copying a smaller image onto a larger image, like a logo copyright on a photo, for example.

So there you have it. Although I believe the first method of layering the images via CSS works just fine, this method will work when you need to deliver a single image.

No related posts.

More Examples with Parsing NDFD Data in PHP

Andrew — Tue, 03 Mar 2009 11:00:10 +0000

I covered how to Parse Weather Forecast Data (from the NDFD) in PHP in a previous article, but due to the amount of questions I received, I decided to show some more usage techniques and examples. In this article, I will cover the time-series option as well as some methods on how to make some data more presentable.

Fetching Time-Series Data from the NDFD

This was covered as an appendix in the previous article, but I will include it here as well in case you are only reading it in the feeds or by email.

Requesting the time-series data returns a whole bunch more information. As requested, here is an example on how to fetch it. Change the desired parameters to ‘true’. The other examples on how to parse the time layouts and formatting data works on this XML, too.

Run This Example

 'time-series',
					'latitude'  => 41.879535,
					'longitude'	=> -87.624333,
					'weatherParameters' => array(
					
	'maxt' => true,			'mint' => false,		'temp' => false,			'dew' => false,	
	'appt' => true,			'pop12' => false,		'qpf' => false,				'snow' => false,	
	'sky' => false,			'rh' => false,			'wspd' => false,			'wdir' => false,	
	'wx' => false,			'icons' => false,		'waveh' => false,			'incw34' => false,	
	'incw50' => false,		'incw64' => false,		'cumw34' => false,			'cumw50' => false,	
	'cumw64' => false,		'wgust' => false,		'conhazo' => false,			'ptornado' => false,	
	'phail' => false,		'ptstmwinds' => false,	'pxtornado' => false,		'pxhail' => false,	
	'pxtstmwinds' => false,	'ptotsvrtstm' => false,	'pxtotsvrtstm' => false,	'tmpabv14d' => false,	
	'tmpblw14d' => false,	'tmpabv30d' => false,	'tmpblw30d' => false,		'tmpabv90d' => false,	
	'tmpblw90d' => false,	'prcpabv14d' => false,	'prcpblw14d' => false,		'prcpabv30d' => false,	
	'prcpblw30d' => false,	'prcpabv90d' => false,	'prcpblw90d' => false,		'precipa_r' => false,	
	'sky_r' => false,		'td_r' => false,		'temp_r' => false,			'wdir_r' => false,	
	'wwa' => false,			'wspd_r' => false)
	
					);

try
{
	/* create the nuSOAP object */
	$c = new nusoap_client('http://www.weather.gov/forecasts/xml/DWMLgen/wsdl/ndfdXML.wsdl', 'wsdl');
	
	/* make the request */
	$result = $c->call('NDFDgen', $parameters);
}
catch (Exception $ex)
{
	/* nuSOAP throws an exception is there was a problem fetching the data */
	echo 'failed';
}

header('Content-type: text/xml');
echo $result;

/* ?> */

Merging Sets of Data with Different Time Layouts

The different sets of data in the NDFD are most likely going to be wanted to be displayed together. For example, on this page, you will see that I have the conditions icons, max temps, and min temps all showing at the same time intervals. But how is that possible?

In this example case, we are going to match up the high temperatures with their conditions icon. This may not give an accurate weather condition for the day because we are going to pick the icon that is the closest to the time stamp given for that temperature. In other words, the condition icons are for every hour or so, and the temperatures are only twice a day. We have to pick two of those icons to match up with the two temperatures. So for example, if the forecast for the day is “sunny in the morning, then t’storms in the afternoon”, the icon will show up as sunny.

Run This Example



NDFD Usage Example w/ Different Time Layouts

Show High Temps with Icons
 $temp): ?>


" />

High:

In the above example, the array_next() function is the one that allows us to match up the data arrays. The first parameter is the array of the weather data that we are trying to find…or in our case, the conditions icon. The second parameter is the key that we want to get close to…or in our case, the timestamp for the temperature we are displaying. So, we loop through the temperatures and search for the closest icon for each one.

Collecting and Using the Time Labels

Depending on that kind of data you are requesting, some of the time layouts contain the time labels. I had those time labels displayed in a data array called time_labels in a previous example, but I didn’t explain how I did it. Basically, I wrote a function that sifts through all of the time layouts. If it finds one with a “period-name” attribute, it saves that timestamp and the period name in a data array similar to the other data items (temp, conditions, etc).

Run This Example

xpath("//start-valid-time");
	$times = array();
	
	foreach ($data as $field => $value)
	{
		if ((string)$value['period-name'])
		{
			$index = (string)$value;
			$times[$index] = (string)$value['period-name'];
		}
	}
	
	ksort($times);
	
	return $times;
}

/* working with stale data, but why query for it every time */
$xml = file_get_contents('ndfd_forecast.xml');
$xml = new SimpleXMLElement($xml);

$time_labels = get_time_labels($xml);

header('Content-type: text/plain');
var_dump($time_labels);

/**
 * Why do I comment out the PHP closing tag?
 * See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
 */
/* ?> */

We can apply it with the Merging Sets of Data example, and end up with something like this:

Run This Example



Show Condition Icons with Time Labels

Show Condition Icons with Time Labels
 $label): ?>


" />

So there you have it – more possible applications and examples with this abundant weather forecast data. As with all other advanced articles on phpstarter.net, this is not intended provide you with copy-and-paste code to paste right in your web applications with little or no modifications. You really need to understand how this all works, and adapt it as necessary or rewrite the examples completely based on what you learned here and what you need to use it for. The point is to get you to understand the concepts used here so you can apply them to your web applications. Happy coding!

No related posts.

Parse Current Weather Conditions Data from the NWS in PHP

Andrew — Thu, 26 Feb 2009 12:00:36 +0000

In a previous article, I covered 5 Sources of Free Weather Data for your Site, but did not provide any actual code to use the data. Last week, I covered source #1 and showed how to Parse Weather Forecast Data (from the NDFD) in PHP. For this article, I will show how to parse source #2 – the current weather conditions data as provided by the National Weather Service.

Fetching the Data for One Station at a Time

This is ideal for a website that wants to show the current conditions for one location. Go to the XML Feeds page, and find your station. We need to know the station id for your area, so select your state, and then find the closest station to you.

The example below shows how easy it is to fetch the XML and format it into something we can read.

fetch('http://www.weather.gov/xml/current_obs/' . $station_id . '.xml', $xml_tmp . $station_id . '.xml');
	
	if (!$snoopy->results)
	{
		/* oops */
		return false;
	}
	
	$data = $snoopy->results;
	
	try
	{
		/* convert the XML into a data object */
		$xml = @new SimpleXMLElement($data);
		/* convert that data object into an array, and return it */
		return get_object_vars($xml);
	}
	catch (Exception $e)
	{
		/* we got an empty or invalid XML file */
		return false;
	}
}

$conditions = load_single_station('KLOT', $xml);
var_dump($conditions);

/**
 * Why do I comment out the PHP closing tag?
 * See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
 */
/* ?> */

Although that example gets the job done, we are going to want to do some caching. It’s a bit pointless to be fetching that XML file every time a page is requested. With a little bit of database code, we can make it relatively easy. First, setup a database that your script has access to, and create this table:

CREATE TABLE `Current` (
  `suggested_pickup` varchar(55) NOT NULL,
  `last_update` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  `suggested_pickup_period` smallint(6) NOT NULL,
  `location` varchar(55) NOT NULL,
  `station_id` varchar(10) NOT NULL,
  `latitude` decimal(5,2) NOT NULL,
  `longitude` decimal(5,2) NOT NULL,
  `observation_time` varchar(255) NOT NULL,
  `observation_time_rfc822` varchar(255) NOT NULL,
  `weather` varchar(55) NOT NULL,
  `temperature_string` varchar(55) NOT NULL,
  `temp_f` smallint(6) NOT NULL,
  `temp_c` smallint(6) NOT NULL,
  `relative_humidity` tinyint(4) NOT NULL,
  `wind_string` varchar(55) NOT NULL,
  `wind_dir` varchar(55) NOT NULL,
  `wind_degrees` smallint(6) NOT NULL,
  `wind_mph` smallint(6) NOT NULL,
  `wind_gust_mph` smallint(6) NOT NULL,
  `pressure_string` varchar(55) NOT NULL,
  `pressure_mb` smallint(6) NOT NULL,
  `pressure_in` decimal(5,2) NOT NULL,
  `dewpoint_string` varchar(55) NOT NULL,
  `dewpoint_f` smallint(6) NOT NULL,
  `dewpoint_c` smallint(6) NOT NULL,
  `heat_index_string` varchar(55) NOT NULL,
  `heat_index_f` smallint(6) NOT NULL,
  `heat_index_c` smallint(6) NOT NULL,
  `windchill_string` varchar(55) NOT NULL,
  `windchill_f` smallint(6) NOT NULL,
  `windchill_c` smallint(6) NOT NULL,
  `visibility_mi` decimal(5,2) NOT NULL,
  `icon_url_base` varchar(255) NOT NULL,
  `icon_url_name` varchar(55) NOT NULL,
  `two_day_history_url` varchar(255) NOT NULL,
  `ob_url` varchar(255) NOT NULL,
  PRIMARY KEY  (`station_id`),
  KEY `latitude` (`latitude`,`longitude`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

After that MySQL table is created, have a look at this next example. It’s the same as the last one, but with two new functions: load_single_station_cache() & save_conditions(). The load_single_station_cache() function checks the table for a non-stale record for the specified station. If it finds one, it returns that database record, and no XML file is downloaded. If it doesn’t find one, it calls up load_single_station(), just like the first example, and then saves that data with the save_conditions() function.

Run This Example

fetch('http://www.weather.gov/xml/current_obs/' . $station_id . '.xml', $xml_tmp . $station_id . '.xml');
	
	if (!$snoopy->results)
	{
		/* oops */
		return false;
	}
	
	$data = $snoopy->results;
	
	try
	{
		/* convert the XML into a data object */
		$xml = @new SimpleXMLElement($data);
		/* convert that data object into an array, and return it */
		return get_object_vars($xml);
	}
	catch (Exception $e)
	{
		/* we got an empty or invalid XML file */
		return false;
	}
}

/**
 * Save the weather conditions to the database table.
 */
function save_conditions($conditions)
{
	/**
	 * Get the columns and interect that list with the data array, 
	 * so we don't try to insert some fields that don't exist
	 */
	$query = "SHOW COLUMNS FROM Current";
	$result = mysql_query($query);
	$fields = array();
	
	for ($i = 0, $n = mysql_num_rows($result); $i < $n; $i++)
	{
		$row = mysql_fetch_row($result);
		$fields[] = $row[0];
	}
	$fields = array_flip($fields);
	$conditions = array_intersect_key($conditions, $fields);
	
	/**
	 * Form the data pairs into query format.
	 * Uncomment the echo statement below to see what it's doing.
	 */
	$fields = $values = '';
	foreach ($conditions as $field => $value)
	{
		$fields .= $field . ",";
		$values .= "'" . mysql_real_escape_string($value) . "',";
	}

	/* remove the last comma from both variables */
	$fields = substr($fields, 0, strlen($fields) - 1);
	$values = substr($values, 0, strlen($values) - 1);
	
	$query = "DELETE FROM Current WHERE station_id = '{$conditions['station_id']}' LIMIT 1";
	mysql_query($query);
	$query = "INSERT INTO Current ($fields) VALUES ($values)";
	//echo $query;
	$result = mysql_query($query);
	if (!$result) die(mysql_error());
}

/**
 * Load the weather conditions from the DB table, and make the necessary 
 * calls if a recent record does not exist.
 */
function load_single_station_cache($station_id, &$xml)
{
	$station_id_esc = mysql_real_escape_string($station_id);
	
	/* no results will be returned if there is no record, *or* if 
		the record is more than 5400 seconds (90 minutes) old */
	$query = "SELECT * FROM Current WHERE station_id = '$station_id_esc' && last_update + 5400 > NOW() LIMIT 1";
	$result = mysql_query($query);
	if (!$result) die(mysql_error());
	
	if (mysql_num_rows($result) == 0)
	{
		/* load a fresh set of conditions */
		$conditions = load_single_station($station_id, &$xml);
		
		/* don't forget to save it for later */
		save_conditions($conditions);
		
		return $conditions;
	}
	else
	{
		/* we got it */
		return mysql_fetch_assoc($result);
	}
}

$conditions = load_single_station_cache('KLOT', $xml);
header('Content-type: text/plain');
var_dump($conditions);

/**
 * Why do I comment out the PHP closing tag?
 * See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
 */
/* ?> */

Fetching the Data for all Stations

Higher traffic sites that show conditions for all or most of the available stations may want to download all of the stations at once from their data feeds page, and import them into a database. To do this, we are going to need to download the zip file from the website, and then extract it. I am not going to provide specific code, because it’s unique to the server environment. Binary files can be downloaded via cURL or fsockopen() as an alternative. Once downloaded, use PHP’s Zip functions to extract the files to a temp directory. Zlib is required to open a zip archive in PHP, so if you don’t have it, you can use the shell_exec() function to run the necessary unzip functions in the shell, assuming that your server supports it.

Once you have the XML files at your disposal, use something like the scandir() function and import all files using something like the single-station example.

So there you have it – XML files turned into an easy to manage, cache-able associative array.

If these examples were enough to make your head spin, there are easy alternatives to show current conditions on your website. Check out these free weather stickers that are easier to use, although not as customizable.

No related posts.

Parse Weather Forecast Data (from the NDFD) in PHP

Andrew — Thu, 19 Feb 2009 12:00:44 +0000

In a previous article, I covered 5 Sources of Free Weather Data for your Site, but did not provide any actual code to use the data. Starting with this article, I will post instructions on how to handle this data as well as sample code. For this article, we will start with the National Digital Forecast Database (NDFD) Simple Object Access Protocol (SOAP) Web Service.

Where to Find the Data

See weather.gov/xml for full details and available data. We will be using the 24 Hourly format, which gives us temperatures, cloud cover %, weather type (rain, snow, etc), hazardous conditions, and even weather icons.

Code to Fetch the Data

Below is some sample code that will fetch the data. The lat/lon coordinates need to be changed for your desired location on lines 10-11. Currently, it is set for Chicago, IL.

 'glance',
					'numDays'   => 5,
					'format'    => '24 hourly',
					'latitude'  => 41.879535,
					'longitude'	=> -87.624333);

try
{
	/* create the nuSOAP object */
	$c = new nusoap_client('http://www.weather.gov/forecasts/xml/DWMLgen/wsdl/ndfdXML.wsdl', 'wsdl');
	
	/* make the request */
	$result = $c->call('NDFDgen', $parameters);
}
catch (Exception $ex)
{
	/* nuSOAP throws an exception is there was a problem fetching the data */
	echo 'failed';
}

header('Content-type: text/xml');
echo $result;

/* ?> */

This is by no means a production sample. You are going to want to cache the results somehow so you aren’t querying the NDFD every time you need this information. It is only updated every hour at most, so there is no need to query it more frequent than that.

Understanding the XML Output

It took me a great deal of staring and reading to figure out how this XML output is organized. Have a look at this sample output, as I will refer to this specific example to explain the sections. I recommend loading the sample XML file in Firefox or any other web browser that allows you to collapse and expand the XML tree.

First, collapse the “head” area by clicking on the minus next to the “head” tag. Everything in there is pretty self-explanatory. Our focus is going to be in the “data” tag. Next, collapse all tags inside the “data” tag, except for the “parameters” tag, but collapse everything inside the “parameters” tag so we can see everything in there w/o scrolling. Here is what your view should look like:

Now we are at a point where we can see how this is all grouped together. As we can see, the “parameters” section shows the products that are available, which are temperature (minimum), temperature (maximum), cloud-amount, weather, conditions-icon, and hazards. Each of them is associated with a time layout because most products are on a different time scale. For example, max temps are during the afternoon & once a day, min temps are at night & once a day, the conditions icons are at several points throughout the next several days, etc. It is separated out this way because not all products have a different time scale – some are the same, like “weather” and “conditions-icon” in our example.

Time to match each product to their time layout. We see that the maximum temps have the time layout of “k-p24h-n7-1″, so we go up to the time layouts and find one with the layout-key of “k-p24h-n7-1″.

We have to code a way to merge all data sets with their corresponding time layout. This may look complicated, but I will show you a somewhat easy way to parse this in PHP.

Run This Example

 $value)
	{
		$arr[] = (string)$value;
	}
	
	return $arr;
}

/*****************************/
/* actual script starts here */
/*****************************/

/* load our example from http://phpstarter.net/samples/348/ndfd_forecast.xml */
$xml_data = file_get_contents('ndfd_forecast.xml');

/* parse the XML data into a giant data object */
try
{
	$xml = new SimpleXMLElement($xml_data);
}
catch (Exception $ex)
{
	/* the XML was probably invalid */
	die('Failed to parse the XML');
}

/* all time layouts go in here with the layout-key as the array keys */
$times = array();

/* loop through the time-layouts in the XML */
foreach ($xml->xpath('//time-layout') as $field => $value)
{
	/* get the layout-key to make it the array key value */
	$layout = (string)$value->{'layout-key'};
	
	
	$times[$layout] = array('start' => get_values($value->xpath('start-valid-time')), 
							'end' => get_values($value->xpath('end-valid-time')));
}

header('Content-type: text/plain');
var_dump($times);


/**
 * Why do I comment out the PHP closing tag?
 * See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
 */
/* ?> */

Now we have all the time layouts in a way we can manage it. The next task is to match it up to the data to their respective time layouts. This next example includes the code to organize the time layouts, and then organize the data to correspond with the times. Running the example below will show that the data is now neatly organized in a way that it can be easily used.

Run This Example

 $value)
	{
		$arr[] = (string)$value;
	}
	
	return $arr;
}

/**
 * Combine a time layout with a product
 * 
 * @param string $data_xpath the xpath to the data set
 * @param string $time_xpath the xpath to the time layout
 * @param array $times the big assoc array of all the time layouts
 */
function merge_times_data($data_xpath, $time_xpath, $times, $xml)
{
	$data = get_values($xml->xpath($data_xpath));
	$time_layout = $xml->xpath($time_xpath);
	if (!$time_layout) return false;
	$time_layout = (string)$time_layout[0]['time-layout'];
	$data = array_combine($times[$time_layout]['start'], $data);
	
	return $data;
}

/**
 * get the time layouts in an array form
 * 
 * @param string $xml
 * @return mixed
 */
function get_time_labels($xml)
{
	$data = $xml->xpath("//start-valid-time");
	$times = array();
	
	foreach ($data as $field => $value)
	{
		if ((string)$value['period-name'])
		{
			$index = (string)$value;
			$times[$index] = (string)$value['period-name'];
		}
	}
	
	ksort($times);
	
	return $times;
}

/*****************************/
/* actual script starts here */
/*****************************/

/* load our example from http://phpstarter.net/samples/348/ndfd_forecast.xml */
$xml_data = file_get_contents('ndfd_forecast.xml');

/* parse the XML data into a giant data object */
try
{
	$xml = new SimpleXMLElement($xml_data);
}
catch (Exception $ex)
{
	/* the XML was probably invalid */
	die('Failed to parse the XML');
}

/* all time layouts go in here with the layout-key as the array keys */
$times = array();

/* loop through the time-layouts */
foreach ($xml->xpath('//time-layout') as $field => $value)
{
	/* get the layout-key to make it the array key value */
	$layout = (string)$value->{'layout-key'};
	
	
	$times[$layout] = array('start' => get_values($value->xpath('start-valid-time')), 
							'end' => get_values($value->xpath('end-valid-time')));
}

$forecast['max_temps'] = merge_times_data("//parameters/temperature[@type='maximum']/value", 
	"//parameters/temperature[@type='maximum']", $times, $xml);
$forecast['min_temps'] = merge_times_data("//parameters/temperature[@type='minimum']/value", 
	"//parameters/temperature[@type='minimum']", $times, $xml);
$forecast['temps'] = array_merge($forecast['min_temps'], $forecast['max_temps']);
$forecast['icons'] = merge_times_data("//parameters/conditions-icon/icon-link", 
	"//parameters/conditions-icon", $times, $xml);
$forecast['time_labels'] = get_time_labels($xml);

header('Content-type: text/plain');
var_dump($forecast);

/**
 * Why do I comment out the PHP closing tag?
 * See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
 */
/* ?> */

There is another function in that example that associates the times to the time labels that you see in the XML. Some of them has a parameter called “period-name”, which is the day of the week or holiday name, if applicable.

How to Use the Formatted Data

For this last example, I will show you in the simplest terms how to use this data. Remember, we are not pulling data from the NDFD live – this is using the XML from our earlier example, so the temps are for mid-winter in NW Indiana.

Run This Example

 $value)
{
	echo 'High temp for ' . $forecast['time_labels'][$field] . ': ' . $value . "
\n";
}

foreach ($forecast['min_temps'] as $field => $value)
{
	echo 'Low temp for ' . $forecast['time_labels'][$field] . ': ' . $value . "
\n";
}

/**
 * Why do I comment out the PHP closing tag?
 * See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
 */
/* ?> */

So there you have it – an XML parsing nightmare made easy. This method is in my opinion the best way to receive and parse this data from the National Weather Service. It may not be the best way for everyone, so if you have something better to add, please post a comment!

(Added 2009/02/20) Fetching Time-Series Data from the NDFD

 'time-series',
					'latitude'  => 41.879535,
					'longitude'	=> -87.624333,
					'weatherParameters' => array(
					
	'maxt' => true,			'mint' => false,		'temp' => false,			'dew' => false,	
	'appt' => true,			'pop12' => false,		'qpf' => false,				'snow' => false,	
	'sky' => false,			'rh' => false,			'wspd' => false,			'wdir' => false,	
	'wx' => false,			'icons' => false,		'waveh' => false,			'incw34' => false,	
	'incw50' => false,		'incw64' => false,		'cumw34' => false,			'cumw50' => false,	
	'cumw64' => false,		'wgust' => false,		'conhazo' => false,			'ptornado' => false,	
	'phail' => false,		'ptstmwinds' => false,	'pxtornado' => false,		'pxhail' => false,	
	'pxtstmwinds' => false,	'ptotsvrtstm' => false,	'pxtotsvrtstm' => false,	'tmpabv14d' => false,	
	'tmpblw14d' => false,	'tmpabv30d' => false,	'tmpblw30d' => false,		'tmpabv90d' => false,	
	'tmpblw90d' => false,	'prcpabv14d' => false,	'prcpblw14d' => false,		'prcpabv30d' => false,	
	'prcpblw30d' => false,	'prcpabv90d' => false,	'prcpblw90d' => false,		'precipa_r' => false,	
	'sky_r' => false,		'td_r' => false,		'temp_r' => false,			'wdir_r' => false,	
	'wwa' => false,			'wspd_r' => false)
	
					);

try
{
	/* create the nuSOAP object */
	$c = new nusoap_client('http://www.weather.gov/forecasts/xml/DWMLgen/wsdl/ndfdXML.wsdl', 'wsdl');
	
	/* make the request */
	$result = $c->call('NDFDgen', $parameters);
}
catch (Exception $ex)
{
	/* nuSOAP throws an exception is there was a problem fetching the data */
	echo 'failed';
}

header('Content-type: text/xml');
echo $result;

/* ?> */

More tips can now be found here: More Examples with Parsing NDFD Data in PHP

No related posts.

Get the Lat/Lon Location of Major US Cities

Andrew — Tue, 27 Jan 2009 12:00:55 +0000

In this article, learn how to fetch the latitude & longitude location of all major US cities. I’m going to provide you with a comprehensive list of almost 500 major US cities as well as the source, so you can get the updated list as city sizes change.

This data is provided by the National Digital Forecast Database (NDFD). They supply the city data in groups, or all at once, so you won’t be making calls to this service often, as city locations don’t normally change. As a result, you can skip the task of writing XML parsing and import scripts, and just import the data directly from here. I have prepared a SQL import file straight from this data with no alterations.

City Location Data: SQL | CSV

This data is retrieved by calling the LatLonListCityNames() function from the NDFD. The National Weather Service presents the cities in 4 different levels. More information can be found by viewing their NDFD page and scroll down to the the section that describes the LatLonListCityNames() function.

Example: Query the Database for a City Name

The following code is the best way to find the closest city for the lat/lon provided.

Run This Example

*/

Example: Fetching the City Names from the NDFD

You do not need to know this if you use the SQL import file offered above. If you want to know how this data was retrieved, here is an example. It requires a copy of NuSOAP, so download yourself a copy if you don’t have it.

 $level);
	
	/**
	 * couple things can go wrong here; the soap request can fail, or we can 
	 * be sent an invalid XML file, which would cause problems
	 */
	try
	{
		$c = new nusoap_client('http://www.weather.gov/forecasts/xml/DWMLgen/wsdl/ndfdXML.wsdl', 'wsdl');
		
		$result = $c->call('LatLonListCityNames', $parameters, '', '', false, true);
		
		$xml = new SimpleXMLElement($c->return['listLatLonOut']);
	}
	catch (Exception $e)
	{
		/* fail! */
		return false;
	}
	
	/**
	 * parse through the XML fields
	 * run var_dump($xml) to see how the XML data fields are formed
	 */
	$city_name_list = explode('|', $xml->cityNameList);
	$lat_lon_list = explode(' ', $xml->latLonList);
	
	$data = array();
	for ($i = 0, $c = count($city_name_list); $i < $c; $i++)
	{
		$row = array();
		list($row['city'], $row['state']) = explode(',', array_pop($city_name_list));
		list($row['lat'], $row['lon']) = explode(',', array_pop($lat_lon_list));
		
		$data[] = $row;
	}
	
	return $data;
}

/* run the function to list all level 1 cities */
$cities = import_city_names(1);

/**
 * For demonstration purposes, we are just going to dump the data.
 * If this were a production script, we would be putting it in storage (database, etc)
 */
var_dump($cities);

/**
 * Why do I comment out my PHP closing tag?  It's actually not needed!
 * See: http://phpstarter.net/2009/01/omit-the-php-closing-tag/
 */

/* ?> */

Do not use this function every time you need to lookup a city. This information is rarely updated, so it’s best to load it into a database or flat file for storage and retrieval as necessary.

I used this source to search for a weather forecast by city. This data isn’t perfect as it doesn’t cover every town and village, but it covers enough for my purposes. If you know of a free data source that gets more specific, please share in a comment.

No related posts.

Locate the Nearest Radar Station and Display Radar Images

Andrew — Fri, 02 Jan 2009 12:00:24 +0000

Including up-to-date radar images on your website is easier than you might think. In this article, I am going to share where you can get radar images for free for US radar sites and even share some sample code that will fetch the closest radar site by lat/lon and display it on your website.

Using Radar Images from Individual NEXRAD Radar Sites

Most of the radar data that you are going to need can be found here: http://radar.weather.gov/ridge/RadarImg/

The radar images are first sorted by the type of radar images you are looking for. Here is what all those codes mean:

N0R = Reflectivity (most commonly used)
N1P = 1 Hour Precipitation
NTP = Storm Total Precipitation
N0V = Velocity
N0S = Storm Relative Motion
N0Z = Long Range Reflectivity

If you don’t know what the different radar types are, the most common one used (like on TV, etc) is “Reflectivity”, or N0R.

If you start browsing the directories, you will notice that it is the radar data only. No county lines, state lines, rivers, etc. Those maps have to be merged in separately. That data is to be downloaded at this URL: http://radar.weather.gov/ridge/Overlays/. Current severe weather warnings are also available as an additional layer. This data can be found at http://radar.weather.gov/ridge/Warnings/Short/.

Putting the maps together are not hard. They are layered on top of each other with some CSS. Below is an example. You may want to move the inline CSS code to an external stylesheet.

More information can be found here: http://www.srh.noaa.gov/jetstream/doppler/ridge_download.htm

Finding a Radar Site’s Location

So far, this is nothing too challenging. No PHP is needed up to this point, as we are just displaying some images. There are times, however, when one doesn’t know which radar site to use. In a recent project, I had to find the closest radar site based on a lat/lon location, not the station name. In order to do this, we need to know the location of each and every radar site. The NWS provides ‘GIS’ files for each radar site that is supposed to be for radar programs to read. We will use this information to plot a map of radar sites.

Each radar site has it’s own GIS file. If you browse the directory at http://radar.weather.gov/RadarImg/N0R/, you will find a .gfw file for each radar site. You will need to parse every one of those files, based on the specs, found here. Luckily, I already did the hard work, and you can reap the results of my labor by importing the data using either of these two files.

Radar Site Data: SQL | CSV

Note: These lat/lon coordinates are not for the center points of the map. Don’t ask me why, but they are for the top-left corner of the map. If you want the center, you will have to offset the lat/lon by about 3 degrees, as you will see me do in the later examples.

Query for the Closest One

Very rarely will the application know the exact location of the radar site. Most of the time, the application will be provided with a lat/lon pair, and it is up to the developer to find the closest station based on that provided location. This can be accomplished with a little math in a SQL query.

-- (PHP code to provide $lat and $lon variables) --

SELECT *, SQRT(POW(69.1 * (lat - $lat), 2) + POW(69.1 * ($lon - lon) *
COS(lat / 57.3 ), 2 )) AS distance FROM RadarSites
ORDER BY distance ASC LIMIT 1

In addition to the table fields, we have an additional dynamic field called ‘distance’ which represents the distance from the radar sites. Sort the results by distance, and voilla, we have the closest radar site.

Using Radar Maps with IP Geolocation

In a previous article, I covered how to get the location of any IP address. I will combine that with this article and provide an example that will show the radar images for the radar site closest to you. If you are not located in the US, then you will see a radar site somewhere on the US coast.

Run This Example





IP Geolocation Example










_Topo_Short.jpg" alt="" />
_N0R_0.gif"  height="550" width="600" alt=""/>
_County_Short.gif" alt="" />
_Highway_Short.gif" alt="" />
_Rivers_Short.gif" alt= "" />
_City_Short.gif" alt="" />
_N0R_Legend_0.gif" name="conditionallegend" alt="" />
_Warnings_0.gif"  border="0" alt="" />
 

Details on IP Location:


Details on Radar Site Location:

More Ways to Find the Location

There are plenty of other ways to get the lat/lon coordinates needed to find the nearest radar site. If you want to find it by ZIP code, the National Weather Service offers a zip code to lat/lon product as part of their National Data Forecast Database. Lat/lon coordinates can also be retrieved from a street address using Google’s Reverse Geocoding service.

No related posts.

5 Sources of Free Weather Data for your Site

Andrew — Thu, 11 Dec 2008 14:00:11 +0000

The weather has always fascinated me, and I actually almost became a meteorologist instead of a full-time entrepreneur. I recently had a project where I had to pull tons of weather data and display it on a website. It took tons of research to find what I was looking for, so I am presenting all my findings here 1) for my own reference, and 2) for your convenience.

This article requires in-depth know-how of parsing through vast amounts of XML data and text. If you are looking for ways to display weather information on your website w/o knowing any of this stuff, wunderground.com has plenty of weather sticker designs that you can add to your site very easily.

Source #1: National Digital Forecast Database (NDFD) Simple Object Access Protocol (SOAP) Web Service

This source is your best bet for free, public domain, information to display on your website, desktop weather tool, etc. This is going straight to the source, which is the National Weather Service. I always recommend going to them first before some of the commercial sites, like Weather.com or Accuweather, because they get most of their weather data from the NWS anyway. This data feed provides virtually all the raw data needed to form a forecast. It doesn’t include the english-formed verbage you might be used to reading, but it does provide all sorts of data from high temps, to precip chances, to risk of tornadoes.

You can send your request via SOAP, which is the recommended way of doing it. I would recommend using nuSOAP, but you are free to use whatever you like best. There is also a native PHP SOAP library. The NWS also offers the ability to retrieve the information by calling a simple URL with a query string.

Note: You can only query for a forecast by providing the ZIP code or lat/lon pairs, however, the NDFD also provides the ability to query for city names & zip codes and get the lat/lon pairs in return. See this SOAP Response Generator for more information and also the ability to get real responses from the server.

All data is received back in XML format, so you need to have a good grip on parsing XML data.

View the data source here: http://www.weather.gov/xml/

Sample code and usage instructions now available here:
Parse Weather Forecast Data (from the NDFD) in PHP
More Examples with Parsing NDFD Data in PHP

Source #2: XML Feeds of Current Weather Conditions

Another data feed service provided by the NWS, this product provides current conditions from all NWS weather stations. If the current conditions are desired for only a few weather stations, they can be fetched individually. The option is also available to download all current conditions in one zip file.

All data is, again, received in XML format, so you need to have a good grip on how to process XML data. The zip file contains the XML file for every weather station, so some programming is needed to extract the file and import each and every XML file.

View the data source here: http://www.weather.gov/xml/current_obs/

Sample code and usage instructions now available here:
Parse Current Weather Conditions Data from the NWS in PHP

Source #3: Weather.gov/data

This last data source from the national weather service is not well documented, and is very difficult to integrate into any software. If you are looking for those English forecasts that read something like “Monday…Mostly cloudy with a 40 percent chance of light snow. Highs around 30.“, then you need to parse this next data source by hand. We don’t have the luxury of XML here.

Have a look at this link. This is what they call the “Zone Forecast Product” for my local weather station. In order to get the ZFP for your station area, you need to load http://www.weather.gov/data/XXX/ZFPXXX, where XXX is the three-letter code for your station. As you can see, there are several forecasts on that page. That’s because there are many zones in one station area. It’s up to you to find out which one you want.

To understand the zone locations for these forecasts, you need to study the Universal Geographic Code specification very carefully. You must parse that cryptic UGC line that’s at the top of every forecast on the ZFP page. If you want to know the locations of each zone, see the County-Public Forecast Zones Correlation File.

If you’re curious what other data you can mine through, besides the zone forecast data, browse http://www.weather.gov/data/.

Source #4: Weather Underground XML API

Our next data feed is from a non-government organization. Their Wiki now says that their data is free of charge for “personal, non-commercial purposes”. If you are using their data for commercial or for-profit sites, be sure to contact them for licensing. If you are not using this data for profit, it’s one of the easiest data sources to use. All you need to know is how to parse XML and fetch remote pages.

You can’t download current conditions or forecasts in bulk, but you can query by zip code, airport code, or city. With using the NWS data you have to use the zip code, lat lon, zone code, or station number.

Wunderground does provide something that the National Weather Service does not. Wunderground has a vast network of personal weather stations and webcams. So if you query for current conditions, you get the NWS station observations as well as live views of the skyline from people’s webcams and data from personal weather stations.

Source #5: Weather.com XML Data Feed

This one is not my favorite because it has the most limitations. You get location specific information, current observations, and two-day forecasts, but there are several limitations:

Must link back to weather.com
Can’t mix weather.com data with data from other sources
Don’t modify the data (makes sense to preserve their reputation)
Can’t redistribute the weather data
Must display a weather.com logo
Developed application must be free to use/access

These are just a few of them, you will have to check out the data source to see your full list of requirements to use the data. Point is, you will have to go through some legal mumbo-jumbo before you can begin coding your application. The up-side with this source is that it is well documented.

View the data source here: http://www.weather.com/services/xmloap.html

So there you have it. I have provided you with enough data to launch your own weather site. It’s up to you to parse and present the data. The point of this post was to show you the vast amount of weather data that is widely available. Watch our RSS feed for future posts, as I will be posting some implementation code on some of these data sources in the future.

If you have something you would like to see sample code for, or if you would like to see something explained in more detail, please let your voice be heard! (post a comment)

No related posts.