The Cypherpunk Manifestos

24/06/2018

Reading a lot about privacy and anonymity and cryptography and cryptocurrency and Darknet hidden services and Tor lately.  Something that has caught my attention is the Cypherpunk movement, and their manifestos.

bitcoin

Without anonymous currency, we don’t have real anonymity

The earliest one seems to be The Crypto Anarchist’s Manifesto, written by Timothy C May in 1988.  Here’s a link to it.  Written thirty years ago, but very of the moment even now.  Read how it opens:

Computer technology is on the verge of providing the ability for individuals and groups to communicate and interact with each other in a totally anonymous manner. Two persons may exchange messages, conduct business, and negotiate electronic contracts without ever knowing the True Name, or legal identity, of the other. Interactions over networks will be untraceable, via extensive re-routing of encrypted packets and tamper-proof boxes which implement cryptographic protocols with nearly perfect assurance against any tampering. Reputations will be of central importance, far more important in dealings than even the credit ratings of today. These developments will alter completely the nature of government regulation, the ability to tax and control economic interactions, the ability to keep information secret, and will even alter the nature of trust and reputation.

A cypherpunk’s manifesto” by Eric Hughes, is also very relevant, even though it is 26 years old.  Here’s a bit:

Cypherpunks write code. We know that someone has to write software to defend privacy, and since we can’t get privacy unless we all do, we’re going to write it. We publish our code so that our fellow Cypherpunks may practice and play with it. Our code is free for all to use, worldwide. We don’t much care if you don’t approve of the software we write. We know that software can’t be destroyed and that a widely dispersed system can’t be shut down.

Cypherpunks deplore regulations on cryptography, for encryption is fundamentally a private act. The act of encryption, in fact, removes information from the public realm. Even laws against cryptography reach only so far as a nation’s border and the arm of its violence. Cryptography will ineluctably spread over the whole globe, and with it the anonymous transactions systems that it makes possible.

This is stuff that could have been written yesterday.  The technologies required for true anonymity have broken out fairly recently: encryption, cryptocurrency, all this has come to a head now.  If we don’t seize this opportunity, maybe we don’t deserve it.

bmc-yellow


Darknet Part 1: What is the darknet and why should I care?

23/06/2018

 

Welcome to Part 1 of my guide to the Darknet.  Well, I say “mine” but it’s actually by many people.  And, just so you all know I’m not trying to pass off this guide as my own words, I’m going to show the words actually coming out of their true creators’ mouths, thanks to the miracle of video streaming over the internet! Thanks be to Youtube,eh!!

Okay, part 1 of this series is a primer on the Deep Web and the Darknet.  It’s a TEDx talk by Alex Winter (of Bill & Ted fame), entitled “The Darknet isn’t what you think”.  There are some misconception about what illegal services were available through the Silk Road website.  For instance child pornography was banned.  Stolen goods weren’t allowed.  Ads for contract killers weren’t allowed.

Anyway, check out the vid.  Enjoy!

Next time: A film about the rise and fall of the Silk Road

 


Apple closes security loophole in iPhones and other iOS devices

14/06/2018

Today Apple is closing a security loophole in iPhones and other iOS devices that enabled law enforcement to hack into criminals’ devices, inculding one of the San Bernadino killers.

They have introduced “Restricted USB Mode”, which will stop hackers from extracting data through an iPhone’s lightning port an hour after being locked.  It is believed that this is how the FBI were able to read data from the iPhone belonging to a gunman involved in the shootings in San Bernadino.

Apple says this is part of their usual security reviews, and is not aimed at thwarting law enforcement but is to protect users from criminals.

GreyKey-box

The GreyKey device that hacks into locked iPhones via its Lightning port

This will protect iPhones from the iPhone hacking tool GreyKey.

The new default settings will have a feature Apple call a “USB restricted mode” which has been present in developer betas for both iOS 12 and iOS 11.4.1. With this feature, all communication through a Lightning port to USB connection will be blocked on unlocked and dormant devices.

US law enforcement uses a tool called a GrayKey, which is a small box with two Lightning cables that can unlock password encryptions on iPhones and extract data from  iPhones.  The Restricted USB Mode will cut off the GreyKey’s access.

hacked-iphone

The GreyKey device reveals a locked iPhone’s passcode in as little as 30 seconds

Of course the cops believe this is aimed firmly at law enforcement, and will result in criminals and terrorists getting away with serious crimes.

“I think that privacy protections are on a collision course with responsible law enforcement actions to conduct legitimate investigations,” said Ronald Hosko, a former assistant director of the FBI who is now president of the Law Enforcement Legal Defense Fund, which raises money to defend officers accused of misconduct. “Terrorists or other criminal organizations will do something that’s heinous, in a way that is blocked from lawful law enforcement view. They will to some extent get away with it. We will lose lives, we will lose infrastructure in a big way, and then we will be having a different conversation.”

bmc-orange


ibVPN – safe web browsing for not much money

08/06/2018
ibvpn-4616-reviews

ibVPN – a high-rated VPN service with more than 180 servers world-wide

A VPN (Virtual Private Network) is a technology that creates a safe and encrypted connection over a less secure network, such as the internet. VPN technology was developed as a way to allow remote users and branch offices to securely access corporate applications and other resources. Nowadays VPNs are widely used to encrypt and secure an otherwise insecure connection (such as a public wifi access point – an eavesdropper can see everything you do over McDonalds’s wifi if it isn’t encrypted!); some people use VPN service to access restricted online service – eg if you live in the UK you won’t be able to use the US Netflix service as that is geographically restricted to users in the USA.  But if you use a VPN server based in the USA, Netflix won’t be able to tell that you’re not in the USA yourself – all Netflix can see is that your traffic is coming and going from that US-based server.  This feature also lends some anonymity to the internet connection, which is another reason some people use a VPN.

And  it’s not just geographical restrictions that VPN use can help you circumvent: some work and school networks stop users accessing some sites like Youtube for instance (your employer may want you to work rather than look at cat videos) or hacker sites (schools tend to block sites with crime-related content, and as so many people associate hacking with crime, anything containing the word “hacker” gets banned).  So, the local network won’t let you view what you want?  Use a VPN, and all the local net can see is data going to/coming from the VPN server.  It knows nothing about goddamn cat memes or how to crack Facebook accounts!

For the past few years I have been using ibVPN (“Invisible Browsing”), run by Romanian-based service provider Amplusnet.  It’s not the fastest service out there, but it is competitively-priced and has global availability.  ibVPN boasts of more than 180 servers in 47 countries across the globe.  And there are 4 different service plans:

  • Ultimate, at $4.83 per month –  “Great for strong privacy and securityheavy streamingunblocking restricted websitestorrents & p2p activity. The most complete package”
  • Standard, at $3.08 per month – “Great for regular usagestreamingunblocking restricted websitesprivacy protection. Includes access to VPN and Extensions. No SmartDNS.”
  • Torrent, also $3.08 per month – “Special package for those looking to protect their identity while downloading torrents. Privacy protection. No SmartDNS or Proxy.”
  • IBDNS/SmartDNS, also $3.08 per month – “Special package designed for unblocking restricted websitesand heavy streaming. Includes SmartDNS and access to browser extensions. No VPN.”

Their All-In-One client software/apps is available for Windows, Apple MacOS and iOS, and Android devices, and the services are also compatible with Linux, most routers, smart TVs and gaming consoles.  The interface is clean and efficient (see below).

ibVPN-All-in-one-client

ibVPN All-In-One client interface controls your VPN sessions

If you’re thinking of going with ibVPN but want to try before you buy, they offer a 6 hour free trial period.  And they have a 15 day money back guarantee if you’re not satisfied by the service.  This shows they have confidence in the quality of their product.

The speed of some servers/connections is not always great, but it is rarely appalling and the price is excellent.  All in all, a good service – I’ve been using it for some years now, which is the greatest praise any product could get – if I keep paying for something it’s because it’s the best!!  😉

Buy Me A Coffee


The govt need “back doors” to thwart terror attacks? Bullshit: they just need to do their jobs properly.

01/01/2016

Govts everywhere are talking up their need for back-doors in encryption etc by saying how the Paris killers got away with so much because of their encryption opsec skillz… but it turns out their opsec is flaky as shit and backdoors wouldn’t be nearly as useful to the cops as listening to the repeated warnings they’d got from Turkey.

Wired.com reported that “news reports of the Paris attacks have revealed that at least some of the time, the terrorists behind the attacks didn’t bother to use encryption while communicating, allowing authorities to intercept and read their messages…

“Reports in France say that investigators were able to locate some of the suspects’ hideout this week using data from a cellphone apparently abandoned by one of the attackers in a trashcan outside the Bataclan concert hall where Friday’s attack occurred, according to Le Monde. Authorities tracked the phone’s movements prior to the attack, which led them to a safehouse in a Paris suburb where they engaged in an hours-long shootout with the other suspects early Wednesday. These would-be attackers, most of whom were killed in the apartment, had been planning to pull off a second round of attacks this week in Paris’s La Defense business district, according to authorities.”

Other reports indicate that a previous ISIS terrorist plot targeting police in Belgium was disrupted in that country last January because Abdelhamid Abaaoud—suspected mastermind of both that plot and the Paris attacks—had failed to use encryption. He also carelessly left behind a cellphone in Syria, which contained unencrypted pictures and videos, including one now-infamous video showing him smiling from a truck as he dragged bodies of victims through a street.

The killers were guilty of serious OPSEC failures… sometimes they didn’t use encryption at all, sometimes they left plaintext evidence lying round where anyone could find it. But as crappy as the terrorists were, the French cops were worse: Turkish authorities have said they tried to warn French authorities twice about one of the suspects but never got a response.

But Western authorities, notably the US and the Brits, have been complaining that they need their secret back-doors to beat the killers, even suggesting that  “US companies like Apple and Google have blood on their hands for refusing to give intelligence and law enforcement agencies backdoors to unlock customer phones and decrypt protected communications”.

My question for the authorities is this: if encryption products have back doors built into them for law enforcement to use, isn’t it likely that crooks will also be able to use these back doors to steal our personal info, IDs, banking details, our entire fucking lives?  The govt are constantly losing top secret laptops on trains and in taxis, and computer intruders regularly bust into official data centres and make off with piles of sensitive data.  Do the authorities think their new back doors will somehow be magically better than all the fucked up attempts at secrecy and security they’ve tried before?

US-paramilitaries

Also, if the authorities get their way, they will be able to find out anything they want to about us.  Maybe (ha ha) that’s not a big problem right now.  But who knows what changes in governments will happen?  Far-right parties are getting more popular all the time.  And look at US presidential hopeful cock Trump: one press of a button and he’ll know exactly where to go to round up the Muslims he hates and send them to be tortured and killed by his friend Assad in Syria.

Don’t listen to the authorities when they say why they “need” the ability to access every bit of data on us.  They don’t need it.  They want it.  Just as they’ve always wanted new ways to eliminate those they don’t like.

giff-logo-small

CLICK ME FOR FREE SIMS!

free web stat


Want some privacy and security online? Check out ibVPN!

04/07/2015

I’ve been using using ibVPN for a while, and I think it’s great.  In case you don’t know, “VPN” means Virtual Priivate network.  To use Webopedia’s definition:

VPN is pronounced as separate letters and is short for virtual private network.

VPN is a network that is constructed by using public wires — usually the Internet — to connect to a private network, such as a company’s internal network. There are a number of systems that enable you to create networks using the Internet as the medium for transporting data. These systems use encryption and other security mechanisms to ensure that only authorized users can access the network and that the data cannot be intercepted.

At ibVPN they delete their logs after 10 days, which no doubt frustrates the police.  But they need to learn: Not all users of VPNs and other privacy tools are terrorists or drug traffickers.  Using a VPN, or encryption tools like PGP/GPG is like putting a letter in an envelope rather than sending a postcard that anyone can see.  I think having a private life is an essential human right.

In fact, I’ll offer Cameron and his cronies a deal: if they start posting their private emails, texts, Instant Messages and letters on a website for all to read, I’ll stop using a VPN.  I’m not talking about secret government correspondence.  Just their private, personal communications.

We got a deal, Dave?  Hmm, I guess not.


‘We can intercept your Google and Facebook activity all we want, so screw you!’ says UK government

17/06/2014

The British government has for the first time spelt out why it thinks it has the right to snoop on our Google, Facebook and other internet traffic all it wants.

Charles Farr, the Director General of the Office for Security and Counter Terrorism, has made a statement (available here) that claims according to UK law the security services only need to get warrants to snoop on communications from one UK party to another. Traffic to and from services like Google (which includes Gmail) and Facebook are classed as “external communications”, for which no warrants are required.

This is horrendous. The internet is a network of networks, many of which are in other countries. So a large amount of our online activity will be transferred via networks in the USA and other countries even if the activity is practically domestic. If you send an email via Gmail to another UK citizen, the government classes it as an “external communication”. The same will be true of activity on Facebook, Twitter, and a great many other services, even though your intention is to communicate or share with other UK residents. Tempora, the program run by the British snooping agency GCHQ, gathers data and metadata, then shares it with the NSA. This means that practically all our online activities are stored, and can be used in fishing expeditions, even though GCHQ or NSA do not suspect you of any potentially criminal activity. Tempora is a “buffer” which stores internet data for 3 days and metadata for 30 days. GCHQ’s computers sift through all this data, storing anything that is “of interest”, which means that online privacy really is nonexistent. Which is what many of us have assumed for ages (especially after Edward Snowden’s revelations), but now it’s official.

What really exasperates me is that major criminals and terrorists will be taking steps to avoid this already, for example by using a VPN (Virtual Private Network). The real victims of GCHQ’s activities are us ordinary joes who are not engaged in criminal conspiracies but who want privacy (like people who send letters in sealed envelopes rather than postcards). We could encrypt our communications; but how many of us want to do this? and I’ll bet Tempora looks out for encrypted traffic and logs it as suspect.

The law needs changing. But that’s not going to happen. Why would the government give up these powers? So, I’m going to use my VPN account when I go online, and I advise everyone else to do the same. Tempora’s alarms will be set off by my suspicious activity; but if everyone is doing it GCHQ’s systems will overload. I hope. Remember, GCHQ has supercomputers and massive storage facilities. Big Brother, man! 1984 man!

Locations of visitors to this page


free web stat


Privacy really is (nearly) dead

16/06/2013

Should government agencies be able to track your email traffic, internet browsing, physical location, when you have a crap? Pretty redundant question really. As Bruce Schneier has written, the Prism programme, run by the NSA, has been going on for some time… we only know about it because the unbelievably brave whistle-blower Edward Snowden revealed it to carefully chosen journalists, who have blown the lid on the gig. The NSA have been collecting info on whoever they choose – American, British, whoever – for quite some time, with no judicial oversight at all. That means, the NSA has been spying on the entire world, without even having to get a warrant!

I don’t think there’s much we can do about this. Extraordinary rendition, third-party torture, secret prisons run by the CIA all over the world: this is the status quo. The genie is out of the bottle, and it’s pretty much impossible to cram the bastard back in. All we can do is look to our own privacy and that of our friends, as well as we can (and remember: the NSA has been constructing a massive data storing/trawling centre in Utah; so they can collect as much as they like without worrying about storage capacity – they could conceivably spy on all of us, every person with a phone line or other internet connection, anytime, anywhere.

The other thing we can do is support whistle blowers like Snowden, Bradley Manning, Julian Assange… and all the others out there. As Schneier says in his article on the matter (and I really do urge you to follow his blog
):

The U.S. government is on a secrecy binge. It overclassifies more information than ever. And we learn, again and again, that our government regularly classifies things not because they need to be secret, but because their release would be embarrassing.

Knowing how the government spies on us is important. Not only because so much of it is illegal — or, to be as charitable as possible, based on novel interpretations of the law — but because we have a right to know. Democracy requires an informed citizenry in order to function properly, and transparency and accountability are essential parts of that. That means knowing what our government is doing to us, in our name. That means knowing that the government is operating within the constraints of the law. Otherwise, we’re living in a police state.

We need whistle-blowers.

Leaking information without getting caught is difficult. It’s almost impossible to maintain privacy in the Internet Age. The WikiLeaks platform seems to have been secure — Bradley Manning was caught not because of a technological flaw, but because someone he trusted betrayed him — but the U.S. government seems to have successfully destroyed it as a platform. None of the spin-offs have risen to become viable yet. The New Yorker recently unveiled its Strongbox platform for leaking material, which is still new but looks good. This link contains the best advice on how to leak information to the press via phone, email, or the post office. The National Whistleblowers Center has a page on national-security whistle-blowers and their rights.

Leaking information is also very dangerous. The Obama Administration has embarked on a war on whistle-blowers, pursuing them — both legally and through intimidation — further than any previous administration has done. Mark Klein, Thomas Drake, and William Binney have all been persecuted for exposing technical details of our surveillance state. Bradley Manning has been treated cruelly and inhumanly — and possibly tortured — for his more-indiscriminate leaking of State Department secrets.

The Obama Administration’s actions against the Associated Press, its persecution of Julian Assange, and its unprecedented prosecution of Manning on charges of “aiding the enemy” demonstrate how far it’s willing to go to intimidate whistle-blowers — as well as the journalists who talk to them.

But whistle-blowing is vital, even more broadly than in government spying. It’s necessary for good government, and to protect us from abuse of power.

We need details on the full extent of the FBI’s spying capabilities. We don’t know what information it routinely collects on American citizens, what extra information it collects on those on various watch lists, and what legal justifications it invokes for its actions. We don’t know its plans for future data collection. We don’t know what scandals and illegal actions — either past or present — are currently being covered up.

We also need information about what data the NSA gathers, either domestically or internationally. We don’t know how much it collects surreptitiously, and how much it relies on arrangements with various companies. We don’t know how much it uses password cracking to get at encrypted data, and how much it exploits existing system vulnerabilities. We don’t know whether it deliberately inserts backdoors into systems it wants to monitor, either with or without the permission of the communications-system vendors.

And we need details about the sorts of analysis the organizations perform. We don’t know what they quickly cull at the point of collection, and what they store for later analysis — and how long they store it. We don’t know what sort of database profiling they do, how extensive their CCTV and surveillance-drone analysis is, how much they perform behavioral analysis, or how extensively they trace friends of people on their watch lists.

We don’t know how big the U.S. surveillance apparatus is today, either in terms of money and people or in terms of how many people are monitored or how much data is collected. Modern technology makes it possible to monitor vastly more people — yesterday’s NSA revelations demonstrate that they could easily surveil everyone — than could ever be done manually.

Whistle-blowing is the moral response to immoral activity by those in power. What’s important here are government programs and methods, not data about individuals. I understand I am asking for people to engage in illegal and dangerous behavior. Do it carefully and do it safely, but — and I am talking directly to you, person working on one of these secret and probably illegal programs — do it.

If you see something, say something. There are many people in the U.S. that will appreciate and admire you.

For the rest of us, we can help by protesting this war on whistle-blowers. We need to force our politicians not to punish them — to investigate the abuses and not the messengers — and to ensure that those unjustly persecuted can obtain redress.

It must be really scary, blowing the whistle when you see things going on that just shouldn’t be happening. But we have to blow the whistle nevertheless. Otherwise, governments and their corporate buddies will just become more and more untouchable. They will be able to do what they want to whoever they want whenever, wherever and however they want. Is that the world you want to live in?

That very nearly is the world we’re living in. Only the glare of publicity can stop our world evolving into a massive police state. Do you want that?

Locations of visitors to this page


free web stat


How to search the internet 4: Understanding search engine results

12/05/2010

This is the fourth part of my guide on how to search the internet. Part 1 is here, part 2 is here, and part 3 is here. Part 5, about using “advanced operators” is here.

So you’ve used Google or some other web search engine, following the tips I’ve given you in this little series, and you’ve been confronted with “results” that don’t actually seem to be any help whatsoever. And it’s true, often Google comes across as an incomprehensible joke designed to make you feel bad. But don’t fret: Google (and its kind) really don’t want you to run screaming; they want you to use the results to find what it is you’re looking for. Unfortunately, this may involve having to learn a thing or two about how Google works. It may be scary-looking at first glance, but really Google want you to find their results pages easily to comprehend. They want you to return to Google.com every time you want help in finding what you want. It can be a rather intimidating interface the first time you look at a results page: but it is all pretty simple really. You just need to know how to understanding the reams of info Google throws at you. Hopefully, this 4th part of my guide will make it all seem far easier.

First thing first: very often Google will offer you a list of sponsored results that may give you what you’re looking for; but if you click on a sponsored link you will be putting money in Mr Google’s pocket and chances are that link will be useless. Forget the sponsored links: go for the meat and potatoes in the list of real links.

Look at the search results; very often you will find other kinds of info alongside those results. Stuff like:

Suggested spelling corrections: Google may think you typed in your query incorrectly. If you’re no good at spelling, this can be a life-saver. But if you know damn well you typed your query correctly, forget this option;

Dictionary definitions: Are you actually searching for the word/s you mean to search for? Maybe you are, maybe you’re not. Think about it. Spelling can be a right tricky operation;

Cached pages: Google carries a huge number of pages that are not currently up to date. Maybe one of those cached pages may contain the info you need. Worth checking if regular searches are turning up sweet F-all;

Similar pages: Often Google won’t find a page that contains the precise info you want, but it has algorithms to turn up similar results. Have a look at them, you’ve nothing to lose really…;

News headlines: A webpage dealing with your query might be hard to find, but it’s often easier for Google to find news stories on related material. And these news stories may well include links to more relevant info. This can save you a bunch of time searching for that little nugget of info that will give you what you want. Remember: news stories are updated frequently, whereas a static page may never be more relevant. Use those options;

Product search: You want to know something about a particular project name. So search for that project name, add a bit of info on what the product can/is meant to do, and see what turns up. This approach works a lot more than you might think;

Translation: So what you want isn’t available in your mother tongue. But it may well be out there for speakers of other languages. Just think: if you are looking for info on a product released by a Portugese company, what makes you think that info will be in English? Search Portugese sites, using Google’s Translation feature or the other translators offered by search services. These translators are often pretty crap; but at least it’ll give you a good idea of what’s what;

Do book searches: Useful info may not yet be available in articles, but books often contain useful stuff. So it can often be a good idea to do a book search;

Cached pages: When a web page is undergoing a lot of changes, clicking on a Google link to a page might take you to the latest version of that page, which may be missing information that was presented some time before. Sometimes, these changes can happen frequently, so a Google link will not take you to the info that the search results first suggested.

Fortunately, Google will often cache an earlier version of the page. So, let’s say a particular page yesterday contained the info you want; but you go to today’s version of the page no longer holds that info. A problem? Not necessarily. Next to the Google link to the updated page will be a link to a [i]cached[/i] version of the page; basically, a version of the page that Google downloaded and cached before the important info was removed. So you click to navigate to the cached page, and you will find the info as it was before it got removed. Google’s system of caching certain pages helps ensure that the history of the web is respected to a certain extent.

If you want to download a version of a page that existed longer ago (several weeks, or months, maybe even years) you can go to [b]The Wayback Machine[/b] at archive.org. This is a project to archive internet sites the way they were in the past, so the current generation’s “now now now” attitude doesn’t drive the history of internet sites into oblivion. [b]The Wayback Machine[/b] doesn’t promise to archive the internet of the past forever; but it is a very useful project that has a multitude of potential uses. Archive.org, like most such projects, is run by volunteers and is always in need of financial support, as well as more practical support such as providing servers. I’d advise anyone who finds such projects very useful to contribute even just a few dollars.

There’s a lot of info on how to understand Google results, and how to configure the way Google works to it gives you the info you want and hopefully protects your privacy, here: http://www.googleguide.com/category/understanding-results/http://www.googleguide.com/category/understanding-results/. I really advise anyone who’s seriously into using Google as best they can to check out this info. Google really is one of the best resources available online… and it’s free! Let’s make the most of it while we can! Before the goddamn Man tries to take it away from us!

_gos=’c4.gostats.com’;_goa=354450;
_got=2;_goi=2;_goz=0;_gol=’Free hit counter’;_GoStatsRun();
Free hit counter
Free hit counter

var _clustrmaps = {‘url’ : ‘https://ihatehate.wordpress.com’, ‘user’ : 904987, ‘server’ : ‘2’, ‘id’ : ‘clustrmaps-widget’, ‘version’ : 1, ‘date’ : ‘2011-06-30’, ‘lang’ : ‘en’ };(function (){ var s = document.createElement(‘script’); s.type = ‘text/javascript’; s.async = true; s.src = ‘http://www2.clustrmaps.com/counter/map.js’; var x = document.getElementsByTagName(‘script’)[0]; x.parentNode.insertBefore(s, x);})();Locations of visitors to this page


How to search the internet 2: how a modern web search works

29/03/2010

In the first instalment of this guide on how to search the internet, I gave a little history of the search engine: I covered Archie, Gopher, and site directories like the Open Directory Project. Those are the old technologies, all pretty much obsolete now. That brings us to the present day and the modern search engine.

When I write “modern search engine”, I mean web search sites like Google and Bing. Because they all work in pretty much the same way – the only difference seems to be in the algorithms each service uses.

Now I could tell you all about spiders crawling the web and stuff, but I think most of you would just tune out after a couple of lines. So I will give you 2 lovely Youtube videos to watch instead:

The 3 Minute Guide to How Search Works:

A slightly longer video that looks at the subject from the perspective of a webmaster who wants to increase traffic to his site:

Watched them? Good. So now you have the basic idea: little programs called “bots”, “crawlers” or “spiders” are sent out to crawl over the world wide web, following links, and compiling lists of URLs that they consider to contain good information. And how do these mindless software automatons decide that the info is “good”? It all comes down to the algorithms.

It’s Google’s algorithms – the “secret ingredient” – that has made Google the world’s favourite search engine and kept them at the top for so many years. Any coder of sufficient proficiency can create bots to crawl the web; but it’s the secret algorithms that turn a regular bot into a googlebot. And there just hasn’t been another bot that can compete.

At least that’s how it has seemed for some time. Yahoo has a hard core of admirers; Altavista.com has had success mostly due to its “Babel Fish” translation service blowing its rivals out of the water; but it’s only recently that a true contender for the title of Number One Search Engine to step up and challenge Google. That challenger’s name: Bing.

Microsoft has been trying for years to break into the search engine market, with a plethora of products: Live Search, Windows Live Search, MSN Search – they even tried to buy, then made a deal with Yahoo to get that Microsoft name up there with the giants – but nothing was able to make much impact on Google. Then in 2008 Microsoft (following the tried and tested strategy of “embrace, extend, extinguish”) bought a tech company called Powerset and, importantly, its “semantic technology”. Microsoft claim that their improved technology cuts down on the risk of “search overload”, when a user is inundated with millions of barely relevant results – something that can happen when using Google. And Microsoft has used the near-ubiquity of its web browser, by incorporating Bing into Internet Explorer 8. Google is still number one search engine, but Microsoft has certainly made its mark on the territory.

So who’s going to win this battle of the search engines? I think it could still go either way. Google has years of good form and a hell of an online presence; but Microsoft still owns the desktop and the browser. And anyway, someone else might come from the left field and clinch it in the final seconds – Ixquick is a potential outside bet with their whole “ethical privacy” trip; Google’s got the “Don’t be evil” motto but it’s Ixquick who are out there actually being “not evil” (and if privacy is a major concern, don’t forget Scroogle). One thing we should have learnt from IT history is that nothing is set in stone.

I’ll bet you’re thinking “Oh well done Google and Microsoft, give yourselves a pat on the back… but what in hell has any of this got to do with how to use a goddamn search engine?!! I figured it would be useful to cover all this history and present situation stuff. Well, maybe interesting rather than useful… I certainly find this kinda crap fascinating. But you’re right, it doesn’t tell us a great deal about how to use a search engine. So I promise: the next instalment of this howto will actually cover some proper howto material. So keep ’em peeled… you definitely don’t want to miss this!!

_gos=’c4.gostats.com’;_goa=354450;
_got=2;_goi=2;_goz=0;_gol=’Free hit counter’;_GoStatsRun();
Free hit counter
Free hit counter


%d bloggers like this: