Frisnit Electric Industrial Co. Ltd.

March 12, 2012
by Mark Longstaff-Tyrrell
0 comments

Delayplayer – Radio 4 synced to your timezone

There are many things UK expats miss when abroad. But while Marmite and Tunnock’s Tea Cakes can now be found in most corners of the world, the comforting presence of Radio 4 cannot. At least not in the same form as in the UK. You see although Radio 4 has been available live online worldwide for many years and its programmes are available on demand, anyone who’s grown up with it knows that it’s the timing of the programmes that’s important.

Starting with the Radio 4 UK Theme at 0545 (now discontinued) the Today programme at 0600, through to Book at Bedtime at 2245 and finally Sailing By and the Shipping Forecast at 0045, these programmes mark out the day. If you live in California, currently 7 hours behind London, you’ll wake up to The Afternoon Play and go to sleep listening to John Humphries grilling some hapless politician on Today, which just isn’t the same as hearing it while stumbling around the kitchen half asleep at 7am.

In order to address this problem I’ve designed the ‘DelayPlayer’. This provides continuous feeds of Radio 4′s output synchronised with a number of timezones around the world. This is achieved by maintaining a rolling buffer of the last 24 hours of Radio 4′s output and taking delayed streams from various points in the buffer.

The system consists of two main parts. The first connects to the BBC’s live Radio 4 stream and records timestamped chunks of a few minutes to disk. The data is chunked to ensure the player never has to access a file that’s still being recorded and also to allow content older than 24 hours to be deleted easily. The second part is a simple web server which takes a single parameter ‘delay’ and streams the content from ‘delay’ minutes ago. For example, the URL for GMT-5 would be ‘http://localhost:1234/?delay=300′. As this server is a bit experimental it’s exposed to the outside world via a more robust Shoutcast server running in relay configuration for a number of predefined timezones:

For timezones behind London a delay is added as expected. However zones ahead of GMT can’t listen in to the future (if they could I’d keep the racing results to myself), so the streams are synced by applying a delay equal to 24 hours minus their GMT offset. Admittedly this means that listeners in these zones are getting yesterday’s programmes, however the point of this system is to recreate the experience of listening to Radio 4 in the UK, and apart from news bulletins most of the output is fairly time-insensitive.

The streams aren’t perfectly synchronised. Buffering at the server and the client as well as errors in estimating the offsets into the stream files means that the audio you hear may be tens of seconds ahead or behind the actual local time.

See www.delayplayer.com for currently available streams.

As an indication of how important Radio 4 is, it is said that if transmissions ever cease then UK nuclear submarine commanders are to assume the nation has suffered a nuclear attack and should respond with a retaliatory strike. Think of that next time you’re listening to You and Yours.

March 2, 2012
by Mark Longstaff-Tyrrell
0 comments

Beer Pedometer

Seeing the Nike+iPod fitness tools collaboration reminded me of a project I made in 2008, the Beer Pedometer. Whereas the Apple system motivates athletes by providing a fitness history and goals (as well as providing yet another platform for flogging music) the Beer Pedometer provides motivation by calculating your energy expenditure and displaying the equivalent quantity of beer that you may consume after your run to correct the deficit.

This was built using the STM Primer evaluation device which was released to showcase the STMicroelectronics STM32 series of ARM based processors. Amongst other things these contain a 128×128 colour screen and a three-axis accelerometer. It is this accelerometer that is used to count steps by looking for the rapid changes in vertical acceleration characteristic of walking. The Beer Pedometer was submitted to the 2008 STM32 design contest and somehow took 7th place.

The display shows the number of pints earned, current power output (80W), energy expenditure so far (1061.6kJ*) and the number of steps taken so far (434). The measurement in green is your current metabolic equivalent rate (MET) value which is the ratio of your current metabolic rate to your resting rate. In the display shown it is 1.0, the resting rate. This is defined as 1.1622 W/kg (Watts per kilogram of bodyweight). The MET value of common activities has been calculated for the ‘average’ person; sitting is 1.0, slow walking is 2.9 , fast walking is 4.8 and so on.

In order for the device to calculate your energy consumption it first calculates your current MET value. By entering your stride into the app and counting the number of steps taken per second your speed can be calculated. From that your current MET is obtained from a lookup table. From this your power output can be calculated using the formula:

W = 1.1622 * m * MET

Where ‘W’ is the power in Watts and ‘m’ is your mass in kg. Cumulative energy consumption can be calculated by multiplying W by the time in seconds (Watts = Joules per second). A pint of bitter typically contains 753kJ so the number of equivalent pints can be calculated and displayed. Given my colossal bulk of 70kg, I consume 1.1622 * 70 ≈ 81Watts just by sitting here writing this blog, so I don’t actually have to move in order to earn beer. I can just wait…

*1 Joule ≈ 0.239 calories

The STM Primer development board is powered by a NiMH battery and includes a 128×128 colour screen, accelerometer, two LEDs, a buzzer and a button. It also exposes the processor’s unused GPIO lines so you can attach external hardware. There’s a USB port under the processor’s control so you can implement HID and mass storage class devices. Programming is done in ‘C’ and is achieved over a second USB port to a host PC running a free cut down version of the Raisonance ‘RIDE’ IDE. Sadly the IDE only runs on Windows which is why I’ve had to stop developing on this platform.

July 7, 2011
by Mark Longstaff-Tyrrell
0 comments

iPlayer for Kindle

Over the last few years the BBC‘s video on demand service iPlayer (a kind of British Hulu) has moved beyond desktop PCs, to games consoles, mobile phones, tablets, set top boxes and internet connected TVs. But there’s still a segment of the population who aren’t being catered for; those whose primary computing device is an eReader.

To address this I’ve designed a system to automatically convert a terrestrial digital TV transmission, as captured by a DVB card or PVR, to a sequence of static images annotated with the subtitles from the stream that convey the story as a kind of comic strip format. This can be viewed as HTML or PDF, so expanding the number of devices the content can be viewed on.

In this example I’ve converted an episode of Britain’s favourite aspirational soap, ‘Eastenders‘. The technical details of the conversion process are given later on in this post, but for now the basic operation of the system is discussed.

Principle
A basic strategy is to only render the frames where subtitles appear (subtitle frames). This is very simple to implement given a video file and corresponding timestamped subtitle file, and for some sequences this is quite adequate. For example:

However introductions and reaction shots often have no dialogue but may still contain key plot points. These will be missed by using the subtitle strategy alone. In a programme like Eastenders where the characters spend much of their time staring in to the distance in despair, these silent scenes must be included. To cope with this, scene detection software is used to produce a list of the start and end frames of each scene, then a snapshot is taken from the midpoint of each scene (scene frames). For example, here’s a sequence where Phil sneaks a look at somebody else’s mail after they’ve left the room:

To produce a complete rendering of the entire programme, the subtitle frames and the scene frames are combined. Then any scene frames from scenes that also contain a subtitle frame are discarded. A HTML file can now be created that simply presents the remaining frames in order. This HTML file can then be used as is, converted to a PDF for download on to an e-reader, or sent to a printer. This last format is not only for inveterate Luddites, it also allows the content to be consumed in areas where electronic devices are not suitable, for instance on the beach, or in an aircraft during take off and landing. This format also allows distribution via the postal service. With a typical PDF episode weighing in at only 20MB and with offline viewing, this format has a clear advantage over existing mobile iPlayer services.

Here’s the complete programme presented in an embedded player:

Eastenders, BBC1, Tue 31 May 2011 19:30 BST

Press the ‘Play’ button to advance to the next frame, and the other transport buttons to navigate the programme.

Technical details
In order to convert the video, first the DVB stream is demultiplexed in to the audio, video and subtitle streams using ProjectX. The DVB subtitles are actual images of the rendered captions carried as a subpicture stream. These could be used directly, however I had access to a pre-processed subtitle file in XML format. This gave a file with the caption text together with the start time, end time, position and colour of the caption. This was created from the subpicture titles using OCR and so occasionally suffers from misspellings and incorrect caption colours, but the XML format is extremely easy to process. An example XML subtitles file is given in the resources section of this post and more information about DVB subtitling can be found in the ETSI document ETSI 300 743 ‘DVB Subtitling Systems’.

Once the subtitles are available, a list of scenes in the programme is obtained. I used lav2yuv to do this, which works well enough for this demonstration. To use this first convert the MPEG file to MJPEG using ffmpeg:

ffmpeg -i source_video -r 25 -an -vcodec mjpeg destination_video.avi

Then build the scene list using lav2yuv:

lav2yuv -S list.txt destination_video.avi

This produces a file which is simply a list of start and end frames for each scene. Finally a simple PHP script is used to generate the final HTML output which can be passed through wkhtmltopdf to produce a PDF file. The whole process could easily be added to iPlayer’s current automated transcoding chain.

Future improvements

The DVB stream also includes an audio descriptiontrack for the visually impaired which provides commentary about the scenes. Currently this only exists as audio, however if it were available as a timecoded text file similar to the subtitles file it would greatly improve the contextualisation of scenes. As an example, here’s the audio description immediately prior to the scene above with Masood:Example audio description
It would be simpler and more accurate to use the DVB subpicture subtitles directly. Perhaps this can be done with the mplayer -sid option when capturing frames? However mplayer had difficulty capturing specific frames, preferring instead to capture the nearest keyframe.
The scene detection software currently doesn’t work infallibly and can miss some scenes. It also requires the programme be converted to MJPEG before processing which is slow. A more efficient, tunable solution would be desirable.

Here’s the HTML version on a Kindle, inside a lovely case courtesy of Velvetmutineer

Resources
eastenders.xml – subtitles file
scene_list.txt – scene file generated by lav2yuv
eastenders.pdf – final PDF generated by wkhtmltopdf

The embedded HTML player shown above can also be viewed directly on a mobile device at http://www.frisnit.com/dvb2pdf/:

February 22, 2010
by Mark Longstaff-Tyrrell
2 Comments

Generating binaural sounds for immersive audio environments

I’ve recently been involved in a project to create an audio game that uses binaural sounds to create a 3D audio environment. My task was to create tools to convert anechoic mono sound samples into binaural samples that appear to be originating from an arbitrary position around the listener’s head.

What’s the difference between binaural sounds compared to stereo? Stereo is created using two sensors and allow us to determine the direction of a sound by measuring the time delay of the incoming signal. A sound to the left will reach the left sensor slightly earlier than the right and so on. But with two sensors measuring time delay we can’t differentiate between sounds on opposite sides of the sensor axis or out of the plane of the sensors. And yet with only two ears we are all able to accurately identify the location of sounds behind or above us. So where is the extra information coming from?

As we have developed from infancy, we have learned how our heads and ears modify sounds from different directions and so can gain extra directional information. The ears are not omnidirectional, they have a strong forward sensitivity and sounds from behind are attenuated. In addition, this attenuation is not uniform across all frequencies and we learn to associate this unique filtering of sounds with the direction from which they originate. This means that the binaural experience is to some extent individual and we’re all tuned to our own particular characteristics. Your auditory pathways are connected differently to mine. This paper describes an experiment (using ferrets of all things) which demonstrates that localisation ability is reduced when listening to sounds modified by a head/ear response that is not the listener’s. By recording sounds using microphones embedded in a dummy head or inserted into the ear canals of real humans recordings can be made that can recreate this effect when listened back to using headphones. However it is not always convenient to record sounds in this way and, as in the case of the binaural game project, we want to synthesise this effect.

In order to generate binaural effects we need to capture the way that the head and ears modify sound. This can be done by measuring the impulse response (IR) at the ears to sounds from different positions around the listener. The impulse response is the response of a system to an infinitely short, infinitely loud signal (infinite spectrum). In reality such signals are difficult to generate so alternative sources are used such as starter pistols. Pistols are good for characterising large spaces such as halls, but in the case where we’re measuring human responses we can use something much gentler such as a time stretched pulse. This is a single low power tone swept over the range of frequencies of interest. Post processing can compress this sweep and determine the IR as if it had been generated by a wideband pulse. What you end up with is a collection of responses known as ‘head related impulse responses’ (HRIRs), or ‘head related transfer functions‘ (HRTFs) in the frequency domain. These can be used to create a binaural sound from a mono sample by convolving the HRIR with the sample. There’s a nice interactive demonstration of this very powerful technique at the Joy of Convolution and a longer discussion of binaural recording here.

Lots of work has already been done in this area, some of which has been very generously shared. A group at IRCAM (Institut de Recherche et Coordination Acoustique/Musique) have made available a number of HRIR measurements from different individuals that can be used to create binaural effects. I’ve written a Java application to apply these HRIRs to mono sounds to create binaural effects. I’ve found that percussive sounds, rich in high frequencies, are most convincing. You can read more about it and download it below. As mentioned, the binaural experience is different for each individual so it’s difficult to predict if a particular HRIR set will work for everyone. You may need to try a number of HRIR sets from the link above before you find one that works for you.

The audio games are still in the development stage, but I’ll post back when there’s anything I can show you.

Download

binaural-1.3.zip

Run

Either double-click the file BinauralSound.jar or from the command line type

java -jar "BinauralSound.jar"

This method is preferable as it’ll let you see any console messages that are generated (usually exceptions due to invalid sound file formats).

Sound test

Check to see if the Java sound system is working correctly by clicking the ‘Sound test’ button. You should hear a 1kHz tone alternating from left to right. On Linux the Java sound system sometimes complains that the host sound device is busy. This can usually be solved by restarting the Linux sound service. For example, if you’re using ALSA then as root type:

/etc/init.d/alsa force-restart

Generating binaural sounds

Download a .zip file of HRTF measurements (strictly speaking HRIR measurements) from the LISTEN HRTF DATABASE and load it into the application (no need to unzip it). Load a short (<1Mb) sample to be transformed and click ‘Play’. The sample must be 44.1kHz PCM and 8, 16 or 24 bit. It can be mono or stereo but if it is the latter then only the left channel will be used to generate the binaural output

Here’s a mono anechoic sample of a snare drum to get you started: SnareDrum.wav

The application will sweep the sample from 0° (dead ahead) around your head anticlockwise in 15° steps. You can also dump the generated binaural sounds as WAV files by clicking ‘Generate files…’.

Don’t forget that you’ll need to listen to the sounds with headphones to experience the effect.

February 7, 2010
by Mark Longstaff-Tyrrell
0 comments

Human engineering guide to equipment design

I picked up this book last week from a second hand shop for a pound. It’s a compilation of research into the design of machines, systems and environments to maximise the efficiency of those using them. It was sponsored by the US Army-Navy-Air Force Steering Committee and published in 1963, the height of the Cold War.

Despite being a very dry and functional book, it manages to capture the tension of the time simply by being so exhaustively comprehensive. Nothing is left to chance. From the more obvious stuff such as the logical grouping of controls in instrument panels and the dimensions of walkways it also covers the minutiae like the optimum resistance of levers and the best actuation force and throw angle of switches. It’s beautifully typeset with clear diagrams and graphs and lots of lovely tables of data. I sometimes worry how excited this kind of thing makes me.

Much of the book details various human parameters, such as height, weight and limb length based on the ‘average male Air Force personnel’. All this data is circa 1955 when, if you are to go by the illustrations, the average Air Force personnel wore slacks and had a short back and sides. However I was very pleased to see I would have been in the 95th percentile regarding height, a veritable man-mountain. But while there’s lots of data that is now obsolete, there’s just as much that is still valid. The chapter on ‘Human Dynamics’ treats the human as another component in the machine with their own unique system response, while the chapter on ‘Selection of Signals’ addresses those tricky situations when you have to choose between a buzzer, a bell or a klaxon. Rather more applicable to today is the section ‘Intelligibility in Speech Communication’ that details methods to maintain speech quality in noisy channels with limited bandwidth. These problems are still with us today, we just have more sophisticated tools to apply to them.

The book is full of simple illustrations from the text – snapshots of ‘the right way’ to do something. They all make perfect sense when you have them pointed out to you and these techniques will be entirely transparent to those using the devices. But omit them, and you can bet they’ll be the first thing an operator notices.

There are so many great articles in this book, and I’m sure I’ll post some more in the future, but currently this image sums it up for me:

Here’s to comfort and efficiency!

December 27, 2009
by Mark Longstaff-Tyrrell
0 comments

On the Origin of Frisnit

Why Frisnit? Well at the turn of the century in those heady days of the electric internet, the Wild West style URL land grab was at its peak. DotCom was the buzzword and .com was the only top level domain to be seen in. And clearly the shorter your domain name the more important you were. But try finding a meaningful name available with less than ten characters. Speculative domain registration was widespread, with chancers registering the dictionary in the hope of cashing in. So what was the solution for someone who just wanted a short domain name for personal email and a modest (and most significantly, non-profit making) internet presence? Simple. Choose a meaningless word and register that. Enter Roger Irrelevant, illogical hero of Viz comic and source of a steady stream of nonsense. Take one of these quirky utterances and from it create an aspirational brand, a mystique, nay, a very way of life.

December 26, 2009
by Mark Longstaff-Tyrrell
0 comments

Frisnit enters the 21st Century!!!

After over ten years since the format gained popularity, I’ve finally come to the conclusion that the blog is the way forward for frisnit.com. Both in ease of publication and convenience of consumption, the blog provides a well defined structure for the accumulation and discovery of ideas. Who said I was a laggard?

Over the next few days I’ll be adding references to the existing content of frisnit.com and then all new content will be added here. There are lots of very exciting new projects in the pipeline and I’m looking forward to sharing them with you.

May 18, 2008
by Mark Longstaff-Tyrrell
0 comments

Dating strategy optimisation by the analysis of trends from an internet dating site

A popular dating site in the UK lists the ages of the subscribers and the highest and lowest age of the partner that they are interested in. The initial motivation for this project was to discover the optimum age that should be targeted to maximise chances of dating success. However, during the investigation, patterns in the data were observed that could explain the phenomenon of the male mid-life crisis and predict when it is likely to occur for a particular individual. This knowledge could be used to reduce the impact of this difficult period by giving everyone involved adequate time to prepare, or exploited by marketing companies hoping to sell more convertible sports cars.

Data source

This particular dating site was chosen because each user’s record contains their age (a), and the maximum and minimum age of the partner they are looking for (h and l respectively). This age range is referred to as the ‘desired age’ in this discussion. The site is arranged in such a way that this data can be gathered automatically using a Perl script and saved in a tab-delimited text file. From there it can be loaded into most data analysis software packages. Only the three parameters described above were recorded and no personal data was gathered. An example of the data is given in Table 1.

a	l	h
23	24	31
32	28	37
35	30	45
…	…	…

Table 1: Example data

Calculating the desired age range of individuals

500 samples for each age between 20 and 50 were taken of males and females . Towards the extremes of the age range 500 samples were not always available. The average of h and l were taken and the results plotted. These are shown in Figures 1 and 2. The red dots are the average high limit and the blue dots are average low limit for each age. The dots form an almost perfect straight line. The solid red and blue lines are the best fit lines through the data points. These graphs allow an insight into how the ages of partners that males and females are attracted to changes over time.

Fig 1

Fig 2

The graphs can be interpreted as follows. To determine the age range that a person desires, look up their age on the x-axis. Draw a vertical line from here and at the intersections with the low and high limit lines read off the corresponding ages on the y-axis. For example, from Fig 1 the average desired age range for a 30-year old man is between 24 and 32. Working backwards, to find the ages of women who are attracted to 30 year old men, locate 30 on the y-axis of Fig 2 and draw a horizontal line. At the intersections with the low and high limit lines read off the corresponding ages on the x-axis. In this case the range is about 22 to 32. A function to calculate the optimum age range to target when dating is given here.

Fig 3: An outlier

Evaluation of the ‘half your age plus seven’ rule

The ‘half your age plus seven’ rule has long been cited as the minimum age of a woman that a man should date. Fig 4 below shows this rule overlayed on the female desired age chart. As can be seen, this will only be a successful tactic for men aged below about 28, and even then will be far from the optimum value. A more accurate, if less catchy, rule of thumb would be ‘age minus seven, times one point one seven’.

Fig 4

Observations

These graphs reveal several phenomena:

Range of desired age increases with age

The red and blue lines are not parallel and diverge over time for both sexes. While a 25 year old woman will desire a man aged between about 25 and 33 (a range of 8 years), a 45 year old woman will desire a man between about 40 and 52 (a range of 12 years). A similar pattern is seen in the data for males. Basically everyone gets less choosy the older they get.

Desired age range differs between genders

The solid green line shows the path followed by a partner the same age. This makes it easier to visualise the desired age of a potential partner in relation to one’s own age. From this it can be seen that in general, women are attracted to men older than themselves, while men are attracted to younger women. This corresponds well with anecdotal evidence and it is nice to see it confirmed by the data.

Partners may lose attraction over time

The high and low age limits for each sex increase at less than 1 year per year (the gradient of the lines are less than 1). This means that as people age, they will traverse the desired age range of their partner. For a couple who met when they were both within each other’s desired age range, one of the partners will move out of the other’s desired range at some point in the future. For example, let us suppose that a 30 year old man has a 30 year old partner. The partner’s point in the male’s desired age range is now shown by the green line in Fig 1. As time passes, the green line moves closer to the high age limit until it intersects with it when the couple are aged about 46. Beyond this point, while the male is still well within the female’s desired age range (Fig 2) the female is now outside the male’s desired age range suggesting that the male will begin to lose interest and start looking for a younger female. This observation will now be developed in the following section.

Prediction of relationship longevity

The high and low limit lines from each graph can be represented by an equation of the form:

Where m is the gradient of the line and c is its intersection with the y-axis.

To discover the age at which one partner (p) will lose interest in the other partner (q) we must calculate the intersection of the high limit line of partner p and the line traced by partner q’s age. The line traced by partner q’s age has the parameters m=1 and c=Δa, where Δa is the difference in age between p and q (Δa=q-p). Intersection occurs when:

Solving for x gives:

Using the line parameters in Appendix A, the ages at which a particular couple will be attracted to each other can be calculated. This calculation has been implemented in a web page and can be seen here.

Relationship to male mid-life crisis

In the case of females, the point of intersection between their high age limit and the age of a similarly aged partner occurs when the couple are about 146 years old. This seldom occurs except in some Old Testament couples. For males however, this will occur when the couple are about 46 years old. This coincides with the general age of the onset of the male mid-life crisis and may account for some of the behaviour exhibited by men during this period. A mid-life crisis calculator has been developed and can be seen here.

Determining partner compatibility

As discovered above, what you want and what you can get are two different things. Fig 5 shows the male and female desired age ranges plotted as areas on the graph. The intersection of the red and blue areas shows where mutual attraction occurs between the sexes. It is in this area that reproductive effort should be put to maximise the chance of success. Print out this chart and take it to the next party you go to.

Fig 5: Cut out and keep

Appendix A: Line parameters

The parameters for the lines in Figures 1 and 2 are given below:

	m	c
Male low limit	0.6235	5.0147
Male high limit	0.8367	7.5663
Female low limit	0.7758	4.9411
Female high limit	0.9396	8.8608

Table 2