NoCo Train Alert needs your brain cells and creativity.   The north camera has been doing a reasonable job detecting trains during the day.  Train detection at night has been problematic.  ("Night time" meaning "after dark.")  We're seeking ideas for identifying patterns in data, generated by image analysis, between non-train suspects and real trains - with the goal of improving overall train detection accuracy.  The specific request will be laid out after some basics of how things work are established.

How Train Detection Works 
​The train detection portion of Train Alert is a python program that uses OpenCV - an open source library of image analysis software.  The north camera (a Raspberry Pi with integrated camera) reads 2-3 images per second.  After an image is read OpenCV identifies what, if anything, is "new" in the image (something is deemed  "new" if it wasn't seen in the previous 3-5 images).  The size and location of the new item is specified via coordinates of a rectangle drawn around the object. 

OpenCV uses 4 integers to define a rectangle. 

x = # of pixels top left corner of rectangle is from the image's left edge
y = # of pixels top left corner of rectangle is from the image's top edge
w = width of rectangle; measured in pixels
h = height of rectangle; measured in pixels

When something new and interesting (i.e. relatively large in size) is reported in a frame Train Alert reads two additional frames and records the resulting 8 values (4 for each frame).   At this point three frames have been processed, and 3 sets of x, y, w and h values have been collected.

Now it is time to decide whether these 12 collected values are indicative of a train or not.  The frequency a new and interesting object (i.e. train suspect) appears in the frame is quite high - oftentimes numerous times per minute.  Obviously trains don't appear that frequently.  Leaves blowing in the wind, changes in light due to clouds, a car driving in the frame - these and other situations are examples of what can be reported as "new" in the frame.  The challenge is to separate the non-train chaff from what is really of interest:  a train.  

Here is one very clear example of a how a train presents itself during a daytime detection scenario.  The train is traveling left-to-right (northbound) in this data set:

Key things to notice in the data:

• The object width w is growing with each new frame read.
• The top-left corner of the object is at the left edge of the image in every frame read.
• The distance of the object from the top of the image (y) and height (h) of the object both vary to some degree - but not dramatically.

It is easy to see that the above data represents a scenario where a rectangle appears in the left side of the frame, gets progressively larger with each frame, does not change in height, all while remaining anchored at the left edge.

Daytime trains traveling in the opposite direction present similarly clear data:

• An increasing rectangle width w from frame to frame.
• A decreasing x value with each frame as the object progresses from right-to-left in the frame.
• Similar relatively-constant values of y and h.

For contrast the data below represents one (of many) daytime non-train examples:

Note how all values (in this example) show high variance.  Not all non-train scenarios, however, are as easy to weed out as the above scenario.  Some non-train scenarios are more subtle.  Through data gathering and trial-and-error we've determined if all of the following conditions are true we can virtually guarantee a train is in the frame:

• Rectangle width w gets progressively larger from Frame 1 to Frame 2 to Frame 3
• Extreme spread (difference between min and max values) for both y and h are less than 10 pixels
• x values are either 1/ all 0 or 2/ get progressively smaller from Frame 1 to Frame 2 to Frame 3

​
Nighttime Train Detection
Daylight, literally, enables clear vision and, consequently, data that is clear and straightforward to interpret.  At night circumstances change substantially.  Overall the software has more difficulty seeing changes in scenes primarily due to reduced lighting.  Some other aspects of the camera installation make nighttime train detection challenging:

• The camera is a significant distance away from the train track.
• The track is not perpendicular to the camera.  Trains traveling left-to-right (northbound) are moving slightly away from the camera.  Track angle is approximately 110-120 degrees relative to the camera's line of sight.  
• Conversely (and helpfully) trains traveling right-to-left are angled slightly toward the camera.  The train's headlamps, in this scenario, make it easier for the camera to identify movement.

Over recent weeks numerous 3-frame sets of nighttime data have been collected.  These data sets include trains traveling in both directions, various other "vehicle" scenarios (e.g. trucks driving in camera view in the middle of the night) and a myriad of other non-train/non-vehicle scenarios.  Trains traveling right-to-left in the frame are quite predictable.  Establishing an algorithm to capture these scenarios does not appear to be particularly challenging.

The Problem
Nighttime trains traveling left-to-right have been particularly difficult to identify.  As stated previously, the train is moving away from the camera so it is not possible to see the train's headlamps, which results in the camera not having a clear indication of change/movement in the frame.  As a result, data representing trains in this scenario do not look like train data generated for other scenarios.  

We are seeking help in identifying patterns/tests that can be used to identify nighttime left-to-right (northbound) trains.  When looking at non-train and train data, what are those unique characteristics or combination of characteristics that enable a train to be identified while ignoring non-train anomalies?

This is some of the data we've collected for both train and non-train scenarios.

One metric used in other scenarios (such as daytime detection) is Extreme Spread (ES).  An example of the kind of observation being sought:  It appears a left-to-right train is present when all 4 measurements have a very low ES.  This method, however, proved inadequate when the last Non-Train data set was collected.  We're asking people to look at and play with this data to see if you can identify tests we can do in the software to reliably distinguish between train and non-train data.

The above data can be accessed either by downloading this Excel file or by accessing this public Google Sheet.  (Please copy the data to another Google Sheet prior to manipulating the data so others can get clean copies of the data.)  FYI:  We've generally found best detection accuracy occurs when multiple train-positive events occur in a data set- such as growing rectangle width AND changing x measurement, for example; although clearly that particular test will not positively identify a train in the above data.   The key is to find those unique ways of looking at the data and its patterns to filter out the noise but recognize a train.

THANK YOU for helping with this!  If you have observations or suggestions please either leave a comment below or send email to jimhaselmaier@gmail.com.

​Jim

It has been a while……a long while……since I’ve done an update on TrainAlert.  This is not due to lack of activity.  In fact, my lack of updating here is because of the effort that has been going into TrainAlert.  (For the unitiated:  TrainAlert is a solution, currently under development, to identify when a train is approaching the town where I live so subscribers can “get on the right side of the tracks” before the inevitable traffic disruption occurs.)
 
The primary challenges in getting this solution built have been 1/ finding a robust technology set upon which to base the solution and 2/ unlearning my 30 year old programming knowledge and applying modern software techniques.  The first challenge was addressed thanks to a friend who is interested in Computer Vision technologies.  He pointed me to OpenCV, an open source library of image analysis software routines.  The second challenge was addressed with time, Google and helpful people on the internet.  This blog post is not about the upcoming release of the solution per se.  Instead it’s about the stunning capabilities available in OpenCV.  OpenCV is yet another fantastic example of what the open source model can deliver.  So if you’re interested in kind-of-geeky Computer Vision topics……read on.
 
OpenCV can perform a broad set of tasks on digital images.  The images can be still pictures (i.e. photos) or frames from a video camera.  The train detection portion of TrainAlert is built using OpenCV routines, tied together by custom code written in Python to handle a variety of logic and analytic tasks.  

There are three primary OpenCV capabilities that are used to detect passing trains.  
 
Background Subtraction
The first step in identifying a passing train is finding items that might be a train.  Background Subtraction is the OpenCV process/method of identifying new objects in an image…..objects that aren’t normally there.  OpenCV’s background subtraction capability watches video frames over time (it keeps a history of the previous 5 video frames) and then identifies items in the current frame that are new.  It can’t identify what those items are.  They could be a variety of things.  But it can at least identify that something new is now in the frame.  When something new is found in a frame OpenCV then “subtracts out” the background portions of the image.  What is left is an image that contains only foreground objects (i.e. those items that are new). 
 
Contour Identification
Once it is known new objects are in the frame, they must be identified and characterized in some way.  OpenCV can provide size and location information about objects in images.  OpenCV calls these descriptors “contours”.  For example, OpenCV can draw a rectangle around each contour found in an image, and give back the details of those rectangles; most notably the coordinates of the rectangle’s four corners.  (OpenCV will return a list of rectangle coordinates for every contour in an image.)   
 
In the case of finding a train, a way to prioritize which contour might be most interesting is by its size.  Obviously, trains are pretty large objects.  So after knowing there are new objects in the frame, along with the sizes and locations of the foreground items, the TrainAlert program picks the largest frame contour and considers this the most likely "train suspect".  However the found contour is still just a suspect.  This new “large object” could be grass blowing in the wind.  Or vehicles that are visible in the distance.  Or clouds that are making dramatic movements.  Additional analysis needs to be done to determine if there really is a train in the frame or whether this large contour is some non-train anomaly.
 
​Object Tracking
One way to discern a train vs non-train contour is to see if and how it moves.  A train traveling down a track moves in a pretty predictable fashion.  The “big object” needs to be tracked to see if it is moving and, if so, by how much.  This is where OpenCV's Object Tracking capability is useful.  In order to track an object the TrainAlert program gives the OpenCV Object Tracker the coordinates of the large object that needs to be tracked.  OpenCV looks at the color profile found in that contour, and then looks for that same color profile in subsequent video frames coming off the camera.   When OpenCV sees that object, it delivers back to TrainAlert the coordinates for where that object is in that most-recently read video frame.  When this process is completed for a sampling of video frames, a location history will have been collected for where in the frame that object is over multiple video frames. 
 
At this point OpenCV has done all that it can do.  Now it's time for TrainAlert to decide "Is this a train or isn't it a train?"  In a simple and ideal world one looks at total distance traveled in the frame, compares the starting and ending positions, and concludes the train's direction of travel.  Unfortunately, the world is neither simple nor ideal.
 
It's not very easy to decide if there actually is a train or not.  The train is not always identified in the same portion of the frame.  And some non-train objects don't move like a train moves.  Wind blowing tree branches might result in movement data, but the data will likely be lots of "back-and-forth" and will not have consistent progression in the same direction over time.  Car lights in the distance definitely move in a consistent direction, but their movement is not as dramatic as train movement because they are farther away in the frame.  There are other non-train scenarios as well.
 
Here are two examples of object tracking.  The first shows how OpenCV tracked a real train.  The second video shows how OpenCV found "something new" in the frame, tracked it, but it turned to not be a train.

​The trick in building this solution has been figuring out an algorithm that will look at the collected horizontal position values and decide whether the object in question is behaving as a train would.  Unfortunately it's not quite as simple as seeing if every data point shows progression in the same direction. The precision of Object Tracking is not perfect.  Even with actual trains Object Tracking will sometimes report either no movement or "backward" movement for one or two frames.   The tracked "bush" in the second video is easy to disregard, as total horizontal movement is in a very narrow window.  But other items that aren't trains don't behave in this way.  

I’ll spare the reader the gory details, but the bottom line is a variety of train and non-train object tracking data was put into a spreadsheet and patterns were identified.  Those patterns were identifiable with some simple calculations and, as a result, a method for determining train vs non-train scenarios has been developed.  Although these methods, too, are not perfect, current testing is showing an accuracy rate of greater than 90% for correctly identifying passing trains.
 
The train detection portion of TrainAlert is in final testing mode.  Without the OpenCV technology (and the help of others) I don’t believe this portion of the solution would be in the state it is today.  OpenCV is an astonishing technology that makes me very enthused about the types of solutions that are possible in the future.  It will be exciting to see what sort of Computer Vision applications become available.
 
So what is next for TrainAlert?  This article has been only about the train detection portion.  But there also needs to be a “subscriber notification” portion of TrainAlert – the infrastructure to deliver alerts to users.  Another generous friend is helping me on that front.  As that gets solidified more will be written here to describe how that works.  For now we’re marching ahead to finalize the first release – and help the citizens of Northern CO deal with the disruptive train traffic that plagues the area.
 
Jim

​A couple of months ago the folks at Freakonomics did a fascinating podcast called Why Is My Life So Hard?  It discussed what is technically called headwinds/tailwinds asymmetry; a phenomenon that gives us the (mental) illusion of having more troubles than fortunes, because our troubles require our energy and focus.  This week in my world of development of the train alerting system I definitely had fortunes…..and they did not go unnoticed!

Fortune #1:  Technology
This week actually started out as a struggle.  Not only was I having significant issues with camera disconnects from the ZoneMinder software, but experts in the forums didn’t have suggestions for resolving the issue.  I went to my “backup” option of using iSpy for motion detection.  Initial testing of iSpy went very well.  But I also discovered iSpy has the ability to launch actions when motion is detected.  With the help of a blog documenting some PowerShell scripting, along with a reminder from my spouse on how emails can be sent as text messages, I can now send a text to my phone (via gmail) upon the detection of motion.

The transition to iSpy necessitates the acquisition of a 2nd camera.  Discerning northbound/southbound train direction using iSpy will be a two-camera affair, with some additional scripting to tie it together.  

Fortune #2:  New Perspectives & Encouragement
Sam Solt, a former manager and current mentor, reached out to me after I described this project via this blog, expressing interest in what I was doing.  Sam is a career coach focused on the changing nature of the employee/employer relationship.  Sam, like myself, grew up in Ft. Collins.  He knows well the challenges the trains create.  We’ve talked a couple of times about various aspects of this project.  This week he arranged a discussion with his neighbor, who is a Railfan (the term used to describe those who pursue trains as a hobby).  He helped me understand various aspects of the Ft. Collins trains and their behavior in town.  Most significantly, I became aware of an additional user segment for this solution:  Those who seek out trains.  My assumption had been train alerting was going to benefit those that wanted to avoid the train.  But there are people who drive around trying to find trains….to watch them go by….or photograph them.  To these people knowledge of an approaching train is a good thing.

Beta Test Signup
If you are interested in being part of the beta test of the TrainAlert system we’d love to have you!  Please sign  up here.  As a quick reminder:  The purpose of TrainAlert is to notify people when a train is approaching Ft. Collins so the resulting disruption in traffic can be planned for and/or avoided.
​
Jim

​Are you a software developer Interested in helping to ease the impact Ft. Collins train traffic has on its citizens?  If so…..read on.
 
A few weeks ago I described a project I’ve been contemplating and recently began working on in earnest:  A system to alert the people of Fort Collins to approaching trains.  Since the train line bisects the city the 6-8 daily freight trains are quite disruptive.    
 
After publishing that blog post a long-time friend and colleague, Brian Fromme, contacted me.  Brian expressed interest in helping with the project.  He suggested we make this an open source project.  The project fits well with what open source is about:  Having the community work on something that all can benefit from.  So that’s what we’re going to do:  Make TrainAlert an open source project. 
 
If you’re interested in helping with this project please visit the project page on GitHub and sign up as a watcher.  We’re currently working on building a list of tasks that need attention.  We can definitely use your help!
 
Thanks.
 
Jim

Ft. Collins, Colorado (where I live) has had its “15 minutes of fame” a few times.  It has appeared on a couple of “Top Places To Live” lists.  It’s the home of New Belgium brewery, the makers of Fat Tire Ale.  The town is also somewhat-infamously known for having a railroad track, literally, running down the middle of a downtown street.  These trains cause quite the disruption to people and traffic.  A few years ago the local newspaper reported on the situation.  One point made was that the railroad does not have a set schedule, so there is "no way” for people in town to know when a train is going to wreak havoc, which happens 6-8 times each day.
 
No way to alert people?  I beg to differ.  I happen to live on the southern edge of town with an unobstructed view of the tracks.  Ever since I read that article, I’ve wanted to find a way to alert people that a train is approaching so that pedestrians and drivers could respond appropriately.  This would enable people planning to cross the tracks to get on the other side sooner rather than later.
 
About two weeks ago I started to tackle this challenge in earnest.  With the availability of capable and inexpensive security cameras, a solution seemed (and is) pretty straightforward.  I encountered an interesting challenge:  Today’s cameras do a good job of detecting motion, but they don’t detect the direction of it.  I needed a way to determine whether a train was traveling north or south.
 
A few days ago I discovered the security camera software called Zoneminder.  Its motion detection capabilities are robust enough that, with the help of some kind folks in the Zoneminder user forum, I’m well on my way to having a reliable means of detecting, not only a passing train, but also which direction the train is traveling.
 
So stay tuned.  A train alerting system for Ft. Collins is in the works.  (And….since direction of train travel can be determined... a solution for Loveland is in the works too.)

Do you know of other towns that have their traffic disrupted by trains?
 
Jim

"See you’ve been in the sun and I’ve been in the rain” – So Far Away, Dire Straits
 
Cycling in the wind stinks. Tailwinds can trick you into thinking you're really fit. Headwinds are a thief of effort. As a result, I like knowing how much wind I'll encounter for a cycling trail I frequent before driving the 10+ miles to get to the trailhead. It’s no fun to load everything into the car and drive there only to discover the conditions aren't great. Weather Underground delivers “hyper-local” weather information and forecasts that enable me to determine conditions before leaving home. They do this by gathering data from a large network of weather stations. This is a great example of the "Internet Of Things" - a term I find amazingly boring given the excitement of the solutions.

The Weather Underground website and smartphone app deliver content to users based on data collected from three sources:

• ~2,000 Airport Weather Stations operated by the Federal Aviation Administration. 
• ~26,000 Surface Weather Stations operated by the National Oceanic & Atmospheric Administration (NOAA). 
• ~250,000 Personal Weather Stations operated by individuals and businesses. 
  ​A Personal Weather Station (PWS) is a low cost ($100-$200) device that can easily be installed at a home or business. They are readily available from a variety of sources including Amazon. A PWS collects temperature, humidity, rainfall, and other information.

How did the company establish this huge network of Personal Weather Stations? They recognized an opportunity to leverage the proliferation of low-cost weather stations and their ability to connect to the Internet. Weather Underground also made it easy for weather station owners to register and connect their units with the Weather Underground service.  Now all data from someone's Personal Weather Station is available to anyone accessing Weather Underground. This data is also used to provide forecasts for very precise locations. (e.g. What is the weather forecast for my neighborhood?)
 
Creating a Personal Weather Station is easy.  Weather Underground even provides a Buying Guide that identifies a variety of compatible options. Many stations not listed in their Buying Guide will work as well. A quick search on Amazon.com provides a host of very reasonably priced Personal Weather Stations that will connect to Weather Underground. It’s clear that weather station companies see “Weather Underground Compatible” as an important selling feature as it is frequently listed as a product capability.

Weather Underground users can monitor a specific station very easily because all registered stations are placed on a map. That is what I do before leaving on a bike ride. You can simply click on any station you wish to monitor.

​
​According to a friend who has a Personal Weather Station, the installation and configuration is straightforward. Many units run on solar power and/or batteries, so it isn’t even necessary to have AC power available.

The value Weather Underground provides to its users is astonishing and a result of the vast amount of data it collects from Personal Weather Stations. It is a fantastic example of the "Internet Of Things". The term Internet Of Things (#IoT) doesn't come close to communicating the staggering value that can be created by having devices connected on the Internet.
 
​It did not go unnoticed by me that the image above shows a 2 mph wind at the trailhead.  It's time to go for a ride.

​Jim

​The discipline of Product Management is fascinating. Great Product Managers are both analytic and creative. Analytic skills are required to understand and prioritize customer requirements, perform competitive analyses, and understand R&D budget constraints. Creativity is needed to determine cost-effective features that address the user's requirements. A recent experience with a Bosch appliance caused me to label one of their features as one of the most creative I’ve ever seen in any product.
 
My wife and I are in the middle of a kitchen remodel. Bosch dishwashers, as you may already know, are known for being exceptionally quiet. In fact, they are so quiet, Bosch received “complaints” that users couldn't tell whether the dishwasher was running or not. It is an odd conundrum: Users want a quiet dishwasher, but they also want to know the dishwasher’s status.
 
Bosch’s answer? The Product Management and Engineering teams enabled their dishwasher to project onto the floor (via light) the dishwasher’s status, including the time left in the cycle. Such a brilliant idea!  It doesn’t change the outward look of the unit at all, yet it conveys the unit is running and when it will be done.
 
Is there some sort of Product Management “Gold Star” award for incredibly unique and elegant solutions to customer problems? I doubt it. But if there was, and I had a vote, Bosch would win it hands down!
 
What product feature would you award a “Gold Star”? I’d love to hear what has impressed you in the products you use.
 
Jim

A few weeks ago we were at friends’ for dinner. On the back patio I noticed a weather station. I started asking my friend about it. (This happens to be one of the key folks who was instrumental in sparking my interest in technology early in my career. He gave me the opportunity to experience a wide variety of computing technologies. So we frequently talk technology together.) The conversation went down a pretty predictable path. I asked him about this or that capability of his weather station. He not only answered the questions but offered up other things he thought I’d find of interest. As the conversation was wrapping up I said “I know this might be a long shot, but is your station registered with Weather Underground?” He got a big grin and said “YUP!”.
 
I’ve used Weather Underground (WU) for a number of years. They have great methods for packing a ton of information into a single display. WU’s model for delivering hyperlocal weather relies on individuals with their own weather stations who are willing to register their station with WU. Data from those weather stations is then automatically uploaded to WU. Although I became aware of WU’s data gathering model a year or two ago, it was fascinating to actually see a station providing data to the service. It was profound to actually see one of these stations that I've been relying on as a WU user. WU reports they have over 250,000 weather stations providing data to their service.  What a great Internet of Things example:  Take advantage of a device’s ability to gather data, and then contribute that data to create a more robust aggregated set of data for reporting, analytics, etc.
 
But my interest in WU goes beyond the Internet of Things. Another area of interest in my consulting work is business ecosystems. More specifically, business ecosystems that are structured to create the “upward spiral” (as opposed to a downward spiral). The upward spiral is a virtuous cycle that creates more and more success for all parties as the system succeeds together.  Smartphones and apps are a good example:  The more apps that are available in Apple's App Store or Google Play the more attractive (and successful) those phone platforms become.  And the more successful those phone platforms become, the more app developers think of additional apps to develop and contribute to iTunes and Google Play. The success in each area feeds the other area.
 
WU’s approach for gathering hyperlocal weather data was very wise:  Leverage the power of the people who are already interested in weather. These hobbyists enjoy weather to the point they’re willing to spend their own money on a weather station. And by enabling the hobbyist to connect their station to WU they actually enhance the hobbyist’s experience. Those hobbyists are now contributing to a greater good:  Helping people understand the weather around them.  WU gets hyperlocal weather data for the “cost” of creating methods for personal weather stations to report their collected data.  It’s brilliant!
 
Because WU touches on multiple areas of my consulting work, I’ll be doing a multi-post blog series on the WU service. The business and business ecosystem will be reviewed, as well as the weather station / data collection aspects of their service.  WU brings the two together to deliver a fantastic weather service.
 
Are you a WU user?  If so, why do you like it?  Have you tried it and abandoned it?  If so, why?
 
Jim