The Google Glass feature no one is talking about

Although I'm generally pro-Google Glass I think this raises a couple of really important issues. One is privacy, the other is forced participation.

When I think about privacy there are two main aspects. One is the physical -- who, where, and how observation is occurring. The second is temporal locality.

When you're out and about you can generally tell who is observing you, and how. You can see if someone is looking at you, and you can tell when someone is pointing a camera your way (generally speaking, I'm ignoring creepers here).

What changed with digital recording and the Internet is two things -- the ability to cheaply create & archive digital recordings made distribution easier, and the reach of the Internet made it possible to reach audiences much larger than the original real world audience.

e.g. you may fart in a train of 15 people but the subsequent upload to YouTube may reach millions of viewers. You farted in public and felt embarrassed but embarrassment in front of 15 people is different to 10 million, possibly including your friends, family, professional acquaintances etc.

Digital recording & the Internet also changed the temporal aspect of privacy. Without a recording that moment of embarrassment is exactly that - a moment. Maybe a few of your fellow passengers will tell the story to their friends for giggles but after a few weeks most would have forgotten about the incident. But when it's recorded & uploaded to the Internet it will remain there probably forever.

The technology to record what Glass does exists in your smartphone today. What's different with Glass is that it makes surreptitious recording the norm, and its tight integration with Google, leading to the concern that such recordings may be archived & searchable forever more.

The more worrying issue that the linked post raises is that these factors combine to make participation compulsory for all.

The most important Google Glass experience is not the user experience - it's the experience of everyone else. The experience of being a citizen, in public, is about to change.

Just think: if a million Google Glasses go out into the world and start storing audio and video of the world around them, the scope of Google search suddenly gets much, much bigger, and that search index will include you. Let me paint a picture. Ten years from now, someone, some company, or some organization, takes an interest in you, wants to know if you've ever said anything they consider offensive, or threatening, or just includes a mention of a certain word or phrase they find interesting. A single search query within Google's cloud - whether initiated by a publicly available search, or a federal subpoena, or anything in between - will instantly bring up documentation of every word you've ever spoken within earshot of a Google Glass device.

I'm not entirely sure how I feel about this.

On one hand the idea of forced participation is extremely distasteful. People should not be forced to significantly alter their behaviour (e.g. having to wear disguises or not be seen/heard in public) because of Google Glass. That said, society decides collectively what trade-offs it is willing to make for the perceived benefits.

Our experience with Google StreetView, Facebook, and smartphones shows that as a whole we adapt quickly to intrusions upon our privacy, for good or bad. Glass will be no different -- there will be a period of debate as to what is and isn't acceptable followed by establishment of new norms; the question is what those new norms should be.

One option may be for Google to mitigate these issues by mirroring real world restrictions on the digitised data. e.g. if I record a video at 12pm at McDonalds Wynyard they might limit access (including searchability) to those who were physically present at the time.

There are still major problems with that - people need to trust Google to do it correctly (i.e. expect regular lapses in controls because Google is ultimately a collection of fallible humans), and it doesn't address the surveillance concerns since if the evidence exists it can always be subpoenaed.

My brain hurts.

I've been a CommSec customer for ~12 years now. For the most part they are a decent broker -- reasonable brokerage and a service that largely Just Works. A recent change led to the discovery of some peculiarities of their operations that led to this post.

The story begins in 2001 when I signed up for CommSec and they offered (and still offer) a discounted brokerage rate for clients who also opened a Direct Investment Account (CDIA). This was a transaction account offering ATM access + interest rates somewhere between the typical transaction account (zero interest) and a savings account (high interest).

All was good until 2008 when they started advertising a new CommSec Cash Management product comprised of two new accounts -- the CommSec Cash Account (CCA) and the CommSec Investment Account (CIA). I had a brief look at these products but they offered nothing of value except more hassle - instead of one account I would have two!

In 2009 CommSec advised that they were discontinuing the CDIA and to continue receiving the preferred Internet brokerage rate I'd have to sign up for a CCA + CIA. Great, whatever. I signed up like a good little lamby, no dramas except now I had two more sets of BSB & Account Numbers to manage.

Shortly after I went into a CommBank branch to bank a dividend cheque (this was 2009, and actually in 2013 there are still companies that send out cheques). I was told that as the new CCA is a CommSec product CommBank branches don't handle any of their transactions. That's really great as a CommSec customer -- I've been forced off an old product onto a new, lesser product.

Anti-pattern #1 - force your customers onto new, shittier products

Of course, the bank accounts were never an appealing part of CommSec's service, I only got one because it gave me better brokerage rates.

On to 13th December 2012, I received this email:

DAFUQ YO! I seriously double checked the date to make sure it wasn't a repost from 2001.

So let me get this straight. In 2009 CommSec forced all of their existing CDIA holders onto their new CCA+CIA package. Just 4 years later they force those customers back onto a "new" CDIA which is exactly the firetruckin' same as the product in 2009. DAFUQ YO!

CommSec Product Management

But that isn't Anti-pattern #2, no-ho-ho!

Anti-pattern #2 - force your clients to change BSB & Account numbers not once, but twice, in 4 years.

I don't pretend to understand how BSB numbers work but the one thing I do know as ordinary Joe Consumer is that whole banks have been traded without a single one of their customers needing to change account details. Hey CommBank -- you guys even did it with BankWest, how about doing the same magic for your loyal CommSec customers?

Changing BSB & Account numbers is annoying because as a public company shareholder you then need to update those details with various share registrars... which leads us to anti-pattern #3 and #4.

Armed with my new CDIA details I went to Computershare and in a single online form submission I updated my payment details for ~12 holdings. That's awesome and how it should be done.

Anti-pattern #3 is brought to you by Link Market Services.

Translation:

We kinda don't trust our website. We think it's less secure than you hand writing a signature onto some dead wood pulp and sending it to us via the post because we trust all of those intermediaries much more than we do our own website security

Anti-pattern #3 - tell your clients that you don't trust your own website security enough to let them transact online

Last but not least, we return to CommSec for Anti-pattern #4.

I thought that if CommSec is going to force all of their customers to change account details the least they could do is update their own systems to use the new account details. That was too much to ask.

I have a Margin Loan with CommSec which needs to be linked to a transaction account for topups/withdrawals. It is currently linked to my non-existent CCA and should be linked to my new CDIA. It's not.

I emailed CommSec to ask if/when this change would/should occur and received the following:

Thank you for your enquiry regarding our new CDIA account. My name is Louise, and your service specialist.

Yes, we will most certainly have linked your new CDIA to your Margin Loan for future settlements

I hope this information has been helpful. If you have any further questions, please reply to this email and I will be happy to further assist.

DAFUQ YO. Do you think I am emailing you for my entertainment? It's clearly not the case and if you'd taken the 5 seconds to look at my account you would know that.

Anti-pattern #4 - employ customer service representatives who prefer not to operate on facts

I've just setup Serviio (a DLNA media server) on my FreeBSD machine - a HP Microserver N36L which does not have a powerful CPU by any stretch of the imagination. There's no way it would handle on-the-fly video transcoding which Serviio supports.

Fortunately the T-Box supports a wide variety of video & audio codecs so transcoding is not necessary for the vast majority of common codecs in use today.

However the T-Box will not decode DTS audio -- it passes DTS straight through to its optical audio output. Thus if you have your T-Box plugged into a TV that doesn't have an optical audio in you get no sound on any media files using DTS or AC3.

Serviio profiles to the rescue!

I added this profile to my profiles.xml:

<Profile id="T-Box" name="T-Box" extendsProfileId="1">
  <Transcoding>
    <Video targetContainer="mpegts" targetACodec="mp3" aBitrate="384">
      <Matches container="*" aCodec="dca" />
    </Video>
  </Transcoding>
</Profile>

Which says "If the file has dca (DTS) audio then transcode the audio to MP3". And because the <Video> has no targetVCodec attribute the video stream is passed through untouched meaning my HP Microserver's lack of CPU is not an issue.

For everything else the extendsProfileId attribute tells it to inherit from the default Serviio profile which does not transcode anything so the end result is that only DTS audio is transcoded to MP3, everything else is passed through to the T-Box untouched.

Questions that I frequently get from friends, and particularly from my parents are "where would you go for X?" or "how do I know I can trust Y?".

The "where" or "Y" may be a real bricks & mortar shop or organisation, or a website, or even software. There's so much scammy and low quality crap out there it's hard to tell what's trustworthy.

Let's take a real world example that most will think is ridiculous

You're oot and aboot and see a Commonwealth Bank branch (for overseas readers Commonwealth Bank is the largest bank in Australia, actually one of the top 10 banks in the world by market cap).

How do you know it's a real Commonwealth Bank branch?

You might look at it's facade. The signage, painting etc all looks high quality and "right" -- the polished, professional exterior, interior, and decor you expect of a bank. The staff are all professionally dressed.

You might expect that if it wasn't a real branch that someone would have noticed. Between the council applications, joe public, and the real Commonwealth Bank surely someone would notice if a fraudster decided to open a fake Commonwealth Bank branch.

At the very least, you should be able to contact the Commonwealth Bank to confirm if this is indeed a real branch location. Which leads to the question - how do you get in touch with the real Commonwealth Bank? Do you consult the White Pages? Why is the White Pages trustworthy?

Maybe you know a little bit about business and you decide to contact ASIC (overseas readers: the Australian Securities and Investments Commission), and find the Commonwealth's Bank registered place of business.

Hang on - how do you know you should trust ASIC?

At some point you reach a root level of trust. Your parents (likely) taught you to trust some level of Government which deems that ASIC is the source of truth on registered businesses in Australia.

What I'm getting at is that the foundations of what we believe to be real or true fundamentally comes down to the people that we know & trust. They form the root of our trust network.

Mum sent me to school and implicitly told me to trust that what the teachers were teaching is the truth. I met others at school and over a long period of time establish trust in a select few and over the years they become my advisors on everything from dentists to restaurants and everything in between.

Facebook Graph Search is about making that simpler, as well as exposing knowledge that may otherwise be hidden.

It makes it possible to discover knowledge that your friends have without having to ask everyone (not that I'm against conversation but there is a practical limit to how many people you can meaningfully converse with).

I'm on the waiting list for the Graph Search beta but I hope that it will offer quick, straightforward answers to questions such as "which dentists do my dentist friends recommend?". This may seem like a stupid question but many of my dentist friends have specialised and no longer practice everyday procedures like checkups & fillings but I know I want to go to the dentists that they go to.

Similarly with bike shops. I want to know which bike shops my cycling friends go to, or shop at [online].

Sure I could ask them one by one but if Graph Seach makes it possible to find out with a few taps on the keyboard that sure saves a lot of time.

The ultimate question of course is the tradies one. Every time we've needed a plumber we go through this ridiculous series of phone calls to find a friend or friend-of-a-friend who has most recently hired a plumber AND found s/he did a good job. It's one of those services we only need every few years and plumbers are often busy, have moved/retired etc.

For me this is the canonical example of the importance of one's trust network. There are umpteen sites online with reviews and ratings but nothing beats a recommendation from someone you trust (or someone they trust).

This particular SSD had been running fine in my laptop and still had a Windows 7 install on it but had been moved into a PC whose BIOS would not boot it. Attempting to re-install Windows 7 gave the message "Windows cannot be installed to this disk". The more info dialogue box said something about BIOS saying the disk was unbootable.

Long story short the problem was that I had set a hard disk password when the SSD had been in my Lenovo W520.

The BIOS on my PC mobo (a Gigabyte GA-Z77X-UD3H) didn't appear to have an option to set a password so I could unlock the drive so I ended up having to plug it back into my Lenovo to remove the password (by setting it to the empty string).

Another symptom was when attempting to use the OCZ Bootable Toolbox v4.3 (a custom Linux based ISO OCZ provides for firmware updates & secure erase) it would get stuck during boot saying "Waiting as requested ... 12". The 12 is supposed to be 12 seconds, and it should count down to 0 but in my case it just got stuck there.

I knew the drive was not bricked as it was working fine in the Lenovo but I would've expected BIOS to prompt for a password if necessary, not just mark the drive as effectively unusable even though it recognised the drive's presence.

Traffic

The first sign came as YouTube comment notification emails. I'd been getting a few comments here and there as Will Keith (aka bbillyk on YouTube), the star of the video, has a fan base of his own. But on the morning of 10th October the comments started arriving 10-20 minutes apart and by noon they were coming every minute or two. One of them mentioned Smosh and I discovered Bottles beware! was featured on their front page. Nice.

I credit Smosh with sending "Bottles beware!" viral as they delivered about 30k views in the first day, and from there the video went all over the Interwebs - reddit, Laughing Squid, Boing Boing, 4chan, Bodybuilding.com forums, Know Your Meme, Ebaumsworld and many other sites. Within 24 hours of Smosh's discovery the video garnered 350k views.

Check out the overall breakdown of traffic types.

Roughly half the views (986k of 2m) came from the player being embedded on 3rd party sites with the top 5 being facebook.com (8.6%), redditmedia.com (5.7%) (now at reddit.picurls.com), barstoolsports.com (5.3%), jaidefinichon.com (3.7%), notengotele.com (2.9%). What's encouraging about this breakdown is that no site represents even 10% of the total -- virality is not about hitting any specific site (even though sites like reddit do help), it's about broad reach. See the graph & complete figures.

Being an English speaking person it was really cool to see the video appearing on other language sites - e.g. jaidefinichon.com and notengotele.com are both Spanish language websites. You could tell when the video had reached a new language as batches of comments would appear in Spanish, Portuguese and even Russian -- I'm not sure who this guy is but his channel is pretty popular :)

Direct referral traffic (i.e. links to the YouTube hosted "Bottles beware!" video page) was responsible for 260k views with the top 5 sources being reddit.com (37.6%), facebook.com (31.5%), yonkis.com (4.8%), twitter.com (4.4%), and theawesomer.com (1.2%). See the breakdown.

Unlike the embedded traffic there is clear dominance here by reddit & facebook. However as a percentage of overall traffic reddit and facebook represent only ~5% and ~8.3% of measurable direct traffic, respectively. I obviously can't tell from this data if someone discovers the video via facebook or reddit and shares it via other sites/means.

Twitter

I was very excited to see my video tweeted by Kristen Bell and Freddie Wong, and even the Surfrider Foundation tweeted the video. But it's interesting to note that despite them having over 1.2 million combined followers at the time my total Twitter traffic (embedded + referral) is less than 12k. I can't help but think of Twitter as that crazy old guy on the proverbial soapbox yelling at people on the street -- no one's actually listening.

reddit

It's interesting to see how completely arbitrary a post's success can be on reddit. The video was posted to reddit 5 times with varying titles:

The winner was the rather ambiguous "I was told to put this here". That post got 5,985 up votes, 3,844 down votes (total of 9,739 votes), 1,497 comments, and drove over 98k views. So roughly 10% of redditors who saw this post bothered to up/down vote and/or comment. Note that the exact up & down vote numbers are fuzzed by reddit for anti-spam/anti-gaming reasons.

Will Keith did an AMA (Ask Me Anything) on reddit and seems to have a good sense of humour.

YouTube

Another fun aspect of going viral is having others edit the video. One of the earliest edits was this:

My favourite edit would have to be the inevitable lightsaber version :)

And someone always has to do the metal version:

There were several reviews (aside from the awesome Russian guy already linked above). "GangstaHoodNews" did one:

Ray William Johnson (a popular YouTuber):

And as expected some guy had to do fat jokes. The thing I love about this video is that he gets completely panned in the comments. Fat jokes on the Internet? Seriously? Like stealing candy from a baby.

I also received several video responses, mostly crap, but this guy was intriguing:

Curiously, once you get a viral video on YouTube every man and his dog tries to send you their crap video hoping that you'll link to them. It's a real annoyance.

Pics please

Of course, no viral video would be complete without it's own meme images:

The Hero (credit: Know Your Meme comment)

Skeptical 3rd World Kid (credit: quickmeme)

Even cats love him: (credit: imgur)

Demographics

Surprise! The vast the majority of viewers were male.

86.5% male, 13.5% female
full breakdown

Copyright & Monetisation

This video was the confluence of a funny idea and a desire to try out Freelancer. I had never intended to monetise it but once the video went viral I started getting YouTube messages from various parties asking if they could use my video on their TV show, web site and other YouTube channels. October Films were first, wanting to use the video in season 6 of Channel 4's Rude Tube show. Even my former employer Yahoo! reached out to me!

As I had no commercial intentions I said yes to everyone who sought permission. When I discovered that others had ripped the video wholesale and uploaded it to their own YouTube channels without adding commentary or anything of value I filed Copyright Infringement notices with YouTube. YouTube promptly took down the copies so that was cool to see.

Within the first 48 hours I received an intriguing message from Viral Spiral. With zero experience in the online video space I had no idea who they were except that they were offering to manage my YouTube channel now that I had a viral video. There's two parts to their service.

The first part is licensing "Bottles beware!" to interested parties - the same parties that I had happily granted permission to prior to signing with Viral Spiral. :) The general idea is that folks who might use it for TV or other commercial purposes would license the video paying anywhere from a few hundred to a few thousand bucks. I put off writing this blog post as I was hoping to have something to report on this front but so far no one has signed.

The second is YouTube premium partner status through Viral Spiral's partnership with Rightster. Essentially Rightster sells and run "premium" ads on my YouTube channel and I get a 50% cut of that. I put the term "premium" in quotes because I'm not entirely sure what this actually means. All I know is that once I signed the advertising rights over to Rightster I started seeing big brand ads running on "Bottles beware!" - including Australian brands like Bing Lee, and ads for TV shows such as Modern Family.

This has been quite lucrative. I should be receiving over $400 in ad revenue shortly (I'm not sure if I'm allowed to share earnings data but if someone could confirm that is ok I'd be happy to share the monetisable views & CPM breakdowns).

Considering the video cost all of $30 to produce it's a pretty healthy ROI, nevermind the learnings which for me has been the most valuable part of the whole experience.

How did this all start?

It all began on Wednesday 26th September in the sekret nomitor IRC channel:

15:49 <@sh> http://www.youtube.com/watch?v=wyVHyyuhVAY
15:49 <@sh> rofl
15:49 <@sh> his shirt
15:49 < goosmurf> hahahahahahahahha
15:50 < goosmurf> he's slain a lot of water bottles
15:50 < goosmurf> has anyone counted how many?
15:50 <@sh> haha
15:51 < goosmurf> haha
15:51 <@sh> somebody should compile all his vids
15:51 <@sh> and speed them up
15:51 < goosmurf> 0:53 is hilarious
15:51 <@sh> with a counter
15:51 < goosmurf> he looks so happy lol
15:51 <@sh> hahaha
15:51 < goosmurf> haha
15:51 < goosmurf> how many vids has he got
15:51 <@sh> quite a few i think
15:51 < goosmurf> 115
15:51 < goosmurf> fuck i'm not doing that
15:51 <@sh> they arent all bottles though
15:51 < goosmurf> maybe I can try freelancer.com

Total cost to humanity so far?

About 6.66 man years (over 3,500,000 minutes watched).

I love his most recent video if only for his pug T-shirt. Well, and maybe his Cartman-esque demeanor :)

He regularly tests his knives & swords on bottled water and I wondered how many bottles he'd slain during his YouTube career. That led to the idea of a compilation video with just the bottles being cleaved alongside a running tally.

I'd recently played with Sony Vegas Pro which is an awesome video editing package for the clueless video person (i.e. me) but I'm too lazy to watch all 115 videos to find the bottle slaying bits.

A few weeks ago I attended SydStart - a conference for startups. Matt Barrie's talk was definitely one of the better ones that day and I left with the resolution that I'd have to take a closer look at Freelancer.

This zany video concept seemed like the perfect small task to experiment with on Freelancer. I posted my project and one day later I had 5 bids, ranging from $30-$130. I didn't want to pay too much for what is essentially a novelty video so I opted to go with hempsta, a Bulgarian fellow, as he had the best ratings & experience at the $30 mark.

I imagined it would take a fair few hours to watch (or at least skip through) all of the videos and clip out the bottled water bits, and to edit them together with some music. I also presumed hempsta had other things going on in his life besides my novelty video. So I was quite impressed when he presented me with the first edit 6 days later (he informed he that he would be busy so couldn't start work immediately) and it was pretty much spot on what I pictured.

In summary my Freelancer experience was great. I think the Freelancer site itself is really spammy looking -- there's a million popups, alerts, achievements and all sorts of garbage flying in your face every step of the way which gave me the impression that I was being set up to be ripped off -- but the actual service you get out it is pretty cool.

Oh, and I can't recommend hempsta highly enough if you need some video work done.

As an example this raffle offers a 1 in 150 chance of winning a Max's Severed Head, a TF2 hat which is worth roughly 50x the ticket price. In dollar terms the ticket price is around US$0.50, and the hat worth ~US$25. In any case a 66% margin is pretty tidy!

From a former World of Warcraft player I learned that a common gambling game in WoW is the roll game where one player acts as the house, the player gives her $x and using the in-game /roll function to generate a random number from 1-100 the pay back is something like 1-66 = loss, 67-97 = 2x wager, 98+ = 3x wager.

What's interesting about using in-game currencies is that in some cases the link to real world value is not obvious. Not all games provide a direct conversion mechanism between real world dollars and in-game currency so players may perhaps gamble without the mental association with real financial gains/losses.

I also wonder what other real world gambling games have been ported to virtual worlds. How long before we see a Swoopo implemented with in-game currency?

What other gambling mechanisms have people seen in virtual worlds?

Going further - what other seedier real world activities have been or are to be ported to virtual worlds?

There's a virtual crap tonne of research suggesting that virtual currencies is an emerging money laundering threat but I'd be interested to know of real instances of it occurring.

One wonders why they did not do this sooner. Has it taken this long for piracy to become a sufficiently mainstream practice that it's only now a serious concern?

I'd guess one reason for holding up fast-tracking is the problem it creates by fragmenting audiences. If the broadcaster's business model is built around selling advertising for prime time audiences you have unavoidable time zone issues. For example with US Thursday night being Australian Friday morning, to fight piracy does an Aussie broadcaster broadcast the same episode that Friday night? Or wait til the following Sunday or Monday evening when audiences are likely to be larger?

Every time I think about TV piracy I end up with the same conclusion - IPTV. In an IPTV world you could have content originators from anywhere in the world and without the limitations of a fixed broadcast schedule content can be conveniently viewed by anyone, anywhere, at any time. Ads could be targeted specifically to the viewer, not based on some generalisation about a show being popular with a particular demographic. Or the episodes could be viewed sans advertising for a small fee!

As with all transitions the hard question is how do we get from here to there?

I wonder if, unfortunately, the Australian broadcasters' efforts may be too late. Otherwise law abiding citizens have already learned how to pirate TV. Now that they know how to do it, and that it's ad-free, why would they go back to watching FTA or [paying for] Foxtel?

I'd actually argue that it's in the interests of viewers to continue pirating as it will -- by reducing traditional broadcast audiences -- accelerate the transition towards IPTV which will ultimately lead to a wider array of content being legally available.

Would Hulu exist if BitTorrent hadn't been invented?

It's rather disappointing given that anyone who has ever made sushi before knows how much faster it is to just use a traditional roller. But in an Amazon review of the Sushezi the reviewer mentions a "$20,000 sushi robot". o_O

Must. Find. Sushi robot.

Unfortunately it is an extremely disappointing robot. Skip to the 1:40 mark to see the pinnacle of it's performance. Fortunately, the music in the video is great.

Left unsatisfied with these craptastic inventions I continued surfing YouTube and found this compilation of wondrous dexterity:

Oh, and a bonus teh tarik puller. What a showman.

Wotnews works well because they've restricted their corpus to Australian sources. I get a single email each morning with a list of relevant headlines and a short snippet from each article. Problem is that Wotnews is closing down soon.

Google & Yahoo! alerts are pretty noisy because of the fact that they scour the entire web. Besides that, it's apparent that no one at Google actually uses their news alerts service because it's quite obviously broken -- articles that are in their news index don't show up at all in the relevant arlert and it has remained that way for weeks at a time.

So I read with interest Jason Calacanis' most recent missive on his latest experiment, the LAUNCH Ticker.

I share his first 3 desires:

1. Just the facts in as few words as possible.
2. Only cover things you think I would find interesting/important
3. Give me screenshots, graphs and appropriate links as cleanly as possible.

The tricky thing is #2 and consequently how you implement #1 & #3, though I consider the latter two nice to have as no one has solved relevancy (#2) well.

Jason's LAUNCH Ticker works great for him I'm sure but to me it's full of irrelevant crap. That's not his fault obviously, we simply have different interests.

Relevant is hard -- there's the relevant news you know you want and can, for example, specify the right keywords for: people, topics, brands, etc, essentially entities expressable as text. And there's related news that you don't yet know you'd be interested in -- for example a new competitor, a new product, a new industry.

I think the latter though is another nice to have. No one seems to have solved the basic problem that given a list of entities, please give me a concise summary of relevant news.

To give a concrete example, Wotnews turns this:

Into this:

Granted Wotnews is not perfect; there's still a bit of noise (what I would call tech industry trash gossip). But it saves me from scouring 5-10 individual papers, and it picks up the odd relevant blog post.

If there's any bright sparks out there who would like some end-user advice on building such a tool please feel free to get in touch. Over the years I've seen many "personalised news" services come and go and very few of them produce anything useful.

I'm not aware of any service that uses a real entity/knowledge graph to filter relevant news, though I'm yet to try out LexisNexis' services (if anyone has any experience here please share, I'd like to know how well their services work, and the rough pricing).

One interesting scenario occurred after we had recently changed our provider of sports data. The old provider (let's call them ABC) accused the new provider (call them XYZ) of stealing their data. Note that this was back in the day when these providers were literally hiring folks to watch live sport on TV in order to type in the scores that they would then send to us in XML; so anything you could do to minimise that labour cost is a huge win.

ABC suspected XYZ of scraping the scores data off XYZ's website. In order to catch XYZ scraping ABC deliberately published incorrect data on their own website. In one instance they swapped the final scores of a particular NBL game, waited for XYZ (and Y! Sports) to update, and then flicked the scores back to see if the changes were reflected on both sites (they were).

I'm not sure how that eventually panned out from a legal POV but both ABC and XYZ continue to exist today.

Yesterday one of my monitoring scripts alerted me to a teeny tiny quirk on the ASX Company Announcements page:

Notice the "-" under pages. The announcement obviously doesn't have "-" pages, in fact it has exactly one page.

So it made me wonder how many downstream systems parroted that quirk.

Here's CommSec:

And ETrade:

And CMC Markets:

So the answer appears to be an exciting ... zero.

Oddly enough, ASX's own ERN specific announcements page also lists zero rather than "-".

Which suggests that the code on their generic announcements listing is not the same as the code used on the ticker specific pages.

Finally, it led to the pondering of how that pages number is generated. Clearly if it was derived from the PDF it would not be "-" or zero as the PDF has one page. So maybe someone, somewhere, is typing those numbers in?

That may make sense given that up until ~2003 many of the announcements were PDFs of what appeared to be faxed documents; the number of pages could have served as a simple check that all pages were successfully received.

I wish I could find out what happened behind the scenes to create this "-" or zero page datum. My monitoring script has been running on the ASX for 7 years and this is the first time something like this has appeared.