Building A Surfer Trap – Stage 4

Date: 2011.02.21 | Category: Traffic | Response: 0

Building A Surfer Trap – Stage 4.

Here we go with stage 4 in our surfer trap.

We should now have a pretty basic surfer trap set up with consoles on all of the FPA’s all of the links on all of the FPA’s (Excluding the FPA graphics) should go to either another FPA or, the a POTD Program.

As mentioned in the last stage we are going to start building up some traffic to our surfer trap even though, we are not finished yet.

The first place where you can get some good free traffic would be from a toplist counter.

A good counter that I would recommend would be one of the following:

Outster http://www.outster.com (A Gay Specific Counter)
SexTracker http://www.sextracker.com (A General Adult Counter)

You need to sign up to ONE of the above counters and only ONE.

Usually a counter will ask you for some details when signing up for them these will usually include:

Email Address.
Site Url
Name

For the email address, enter a REAL email.
For Your Name, Enter a name you wish to be known by to the counter owners.

However, for the URL (The one to SEND traffic to FROM the counter) you should enter the index.html page of your surfer trap (The Multi-Site FPA).

You will now be given some coding once you complete the counter sign up process. This counter code you now need to place on EVERY page of your surfer trap.

Place the counter towards the bottom of your FPA’s (Including the Multi-Site FPA) this way, they will load after the rest of the graphics or at least, they should in theory.

Basically what you are doing by implementing this counter is as follows:

For each time the counter is displayed or clicked on your site (Depending on the counter you choose) you will earn one credit on the respective counters toplist.

Each credit you earn on the toplist will move you higher up the list.

Now, the higher up the list you get, the more visitors to the counter site that you are going to get visiting your surfer trap by clicking your link on the toplist.

In return, by clicking on your surfer trap link from the toplist, they are ‘registering’ another impression of your counter hence, getting you one more ‘credit’ on the counter toplist hence, moving you higher in the listings.

See where we are going with this?

The more clicks you get from the counter to your hub, the higher you will get on the toplist in the process earning you more and more traffic to filter through your surfer trap.

Adding these counter codes to your FPA’s as they stand at present should take you an hour or so to do so, for now this is where I will leave the traffic generation section of this tutorial.

Article written by Lee

Fonts – Everything You Wanted To Know

Date: 2011.02.22 | Category: WebDesign | Response: 0

All web browsers use standard fonts. Mostly two types, one for proportional, one for mono spaced fonts. Proportional (or variable width) characters adapt in width, an “m” uses more space than an “i”. Mono spaced (or fixed width) characters are all equal in width, typewriter style.

Most browsers use “Times” for proportional fonts. This font was originally developed by the London Times news paper. “Courier” is used for mono spaced. This was a very popular font used for mechanical typewriters. Both are normally set to 12 points (1/72nd of an inch).

Serif And Sans Serif.

Both these fonts are so called serif fonts. The French word serif indicates the little strokes at the outer ends of the characters. They are very old, you see them in old gothic handwriting, or Greek and Roman buildings. Partly used for ornamental reasons, partly because the characters are easier distinguished.
It’s a little odd they are used on computer screens. These are by nature quite coarse, which makes serif characters quite grainy and ugly. Sans serif characters generally display a lot better on screens. Sans serif literally translates to “without stroke”. Probably the most popular sans serif font is Helvetica or derivatives like Windows’ Arial.

<FONT FACE=”Arial”>…</FONT>

The FONT Tag.

The html tag for fonts is a somewhat crude instrument. Most word processors let you use any font you like, as long as it’s on your system. And that’s the first big problem in web browsers. You have no control over other systems’ fonts. So you will have to choose a font which is likely to be on any system out there.

<FONT FACE=”Arial,Helvetica,Sansserif”>…</FONT>

Typeface.

The font tag accommodates this by letting you specify several fonts in the FACE attribute. If the first one is not available, the second is used, and so on. The set above is often used. Arial is on all Windows systems, Helvetica on Macintosh, Sans serif on UNIX. The same is true for mono spaced fonts in the line below.

<FONT FACE=”Courier New,Courier,Mono”>…</FONT>

Size.

Word processors let you specify font sizes in points exactly. No such luck in web browsers. There are seven sizes to choose from, denoted 1 (smallest) through 7 (largest). If this SIZE attribute is not used it defaults to 3. I think the default 12 point size is a bit big, so I use 2 for size, which gives you about a 10 point character. Some browsers let you set the overall font size smaller or larger. Which makes this issue even more awkward.

<FONT SIZE=”2″>…</FONT>

There is a nasty bug in some browsers. When using a block of text with size 1, the last line skips a line. This bug can be squashed by putting a <BR> tag immediately after the block of text. If your browser has this bug it will show in the second text below.

There is a nasty bug in some browsers. When using a block of text with size 1, the last line skips a line. This bug can be squashed by putting a <BR> tag immediately after the block of text (with break).

Color.

Fonts can have any color you like, much like the colors in the body tag. Keep readability always in mind, avoid clashing colors and little contrast. You can create nice shading effects. But don’t make a Christmas tree out of your page by using too many colors.

<FONT COLOR=”red”>…</FONT>

Style Sheets.

There is a chance all this soon will be replaced by style sheets. They do let you specify exact point sizes, even use downloadable fonts. But for now I would advise against that, since not all current browsers understand them. You could however use a combination of both, should you want to.

Article written by Lee

The Next Big Traffic Craze – Thumbnail Link Lists

Date: 2011.02.24 | Category: Link Lists | Response: 0

As adult webmasters are constantly thriving to create the next craze in traffic sources to garner sales from their sites it comes as no surprise that there is already talk amongst the ranks of the next big thing, what is this new idea? The Thumbnail Link List.

Thumbnail Link Lists – Re-Inventing The Wheel.

As the past year and a half have taught us as webmasters, regular TGP’s have almost become a thing of the past with more and more thumbnail TGP’s being created and submitted to daily, why then, has it taken so long for someone to think up the idea of the Thumbnail Link List? Quite simply I think it has been a case of to much has happened over the last 12 months in the adult industry for webmasters to think this idea up.

So What Is A Thumbnail Link List?

A thumbnail link list is in effect a way of displaying free sites on your own standard Link List by utilizing small thumbnail samples of the images contained within the free sites galleries, in fact, much the same way that a thumbnail TGP works however, instead of only listing single image galleries your link list will consist of links to individual free sites.

Much like the Thumbnail TGP traffic source, the thinking behind the Thumbnail Link List is that by showing your surfers what type of pictures the sites within your link list contain they will be more likely to visit the free site thus improving your CTR to your webmaster submitted sites and, ultimately, your site overall allowing you to manipulate the traffic to garner more sales for yourself as the Thumbnail Link List owner.

Submitting Your Site To A Thumbnail Link List.

Looking at the Thumbnail Link List idea from a webmaster who would be submitting your free sites for listing (and ultimately traffic) how do the rules of the Thumbnail Link List differ to that of the regular link list which we are all used to seeing, well for a start, the main difference will be that you either need to submit a thumbnail yourself or, enter the url to one of your gallery pages in addition to the main entrance to your free site, this will actually be the only significant change in how you operate your link list submissions compared tot he current link list method, however, as each Thumbnail Link List owner will have their own individual rules you should check before submitting as to what is required from you as a submitter.

Thumbnail Link Lists – Overview.

The whole Thumbnail Link List idea is one that, in all seriousness should mean more traffic not only for the submitters of the free sites but, also for the Thumbnail Link List owner themselves, by offering the surfers a preview of the content contained within the free sites they will be visiting they can simply reduce the amount of sites that do not appeal to them and focus more on the sites that do meaning that the webmasters who submit highly niched free sites to the links lists will now be able to benefit from a larger traffic base and ultimately, more sales.

Article written by Lee

Search Engine Optimization, Is It Worth It?

Date: 2011.02.24 | Category: Search Engine Optimization | Response: 0

One of the trendiest takes on Internet marketing these days seems to be the notion that securing top search engine rankings “no longer works.” Where it started, I have no idea.

But rarely does a week go by when I don’t see one or more Internet marketing “experts” claiming that search engine positioning is largely a waste of time and should not be a primary focus of Website owners.

Well, as the saying goes, “there are two sides to every story.”

But let’s not make this article about my side, or your side, or anyone else’s side. Let’s forget about my opinion and other “expert’s” opinions and stick to the indisputable facts, as reported by highly credible third party sources:

According to a Forrester Research Media Field Study, getting a loyal audience in the first place is best done by Search Engine Placement.

According to a GVU Users Survey, 84.8% of Internet users use Search Engines to find Websites.

In a study released by ActivMedia Research in September 1999, Search Engine Positioning was ranked as the #1 Website promotional method used by eCommerce sites.

And look what I just found in the April 2000 issue of Target Marketing Magazine.
“Top Ways Websites are Discovered”

Banner ads: 1%

Targeted email: 1.2%

TV spots: 1.4%

“By accident”: 2.1%

Magazine ads: 4.4%

Word-of-mouth: 20%

Random Surfing: 20%

Search Engines: 46%

You’ve now seen the numbers and know that search engine promotion is very much alive and kicking. But let’s take this a step further. Let’s talk about the *quality* of prospects coming to your Website through search engines as opposed to other advertising mediums.

Every time your potential customers use the search engines, they qualify themselves as *hot prospects* by conducting searches on keywords that are directly related to your product or service. Their choice of keywords is proof that they have a genuine interest in what you offer. These people spend their valuable time exploring the search engines for your type of product or service.

Think About That.

They didn’t stumble upon one of your ads, or wander past a hyperlink to your site. And they didn’t get a banner ad thrown in their face. They made the *decision* to actively search the keywords that brought them to YOUR Website. And when they get there, they are ready and willing to do business with you. At the very least, they’re seriously considering it and, that’s the most powerful sales tool you can have!

But merely getting your Website “indexed” or listed in the search engines is not enough. In order to get any significant traffic from the search engines, your Website must be listed within the top 30 search results (preferably the top 20).

Very rarely will anyone look beyond the first 30 results returned from a search. This makes perfect sense because the most relevant sites are always listed at the top. So if your prospect doesn’t find what they want within the first 20 to 30 listings, they’ll simply do a new search.

If your site falls anywhere below the 30th listing, you don’t stand a chance against anyone in the TOP-20. Hence, it should be your goal to achieve Top 20 positions.

How Do You Get Your Website Listed In The Top 20?

1) You can attempt to gain these Top 20 rankings yourself. However, this can easily become a full time job. (I think this is why so many marketers advise against focusing on search engine positioning.)

2) You can hire a reputable company who can achieve AND maintain your Top 20 rankings for you (be sure they guarantee their service and have several verifiable clients that currently have multiple Top 20 rankings).

3) You can do nothing at all. But as you’ve seen from the third party statistics referenced above, search engine positioning is something you can’t afford to ignore.

Whatever choice you make, I hope that your eyes have been opened to the tremendous profit potential that exists with top search engine placement.

Article written by Lee

New European V.A.T Laws – Are You Prepared?

Date: 2011.02.23 | Category: Billing Solutions | Response: 0

Under a new law that has been passed in the EU from July 1st 2003 any internet based company selling services or goods to customers inside the EU will have to pay the member state of their customer/client the equivalent V.A.T (Value Added Tax) Rate.

This in itself is not a big problem, online business have been paying taxes for many years, the problems start to occur when you take a look at the current state of the V.A.T system in the EU state members on an individual basis.

The following list shows the percentage of V.A.T that you will need to add to your clients bills if they are located inside a member state of the EU:

Austria – 20% VAT
Belgium – 21% VAT
Denmark – 25% VAT
Finland – 22% VAT
France – 19.6% VAT
Germany – 16% VAT
Greece – 18% VAT
Ireland – 21% VAT
Italy – 20% VAT
Luxembourg – 15% VAT
Netherlands – 19% VAT
Portugal – 17% VAT
Spain – 16% VAT
Sweden – 25% VAT
United Kingdom – 17.5% VAT

This will pose several problems for the adult industry namely, how will our billing processors be able to handle transactions from European Union citizens? After all, with so many different VAT rates across the EU their billing systems need to be able to correctly calculate the correct amount of VAT to the surfers final order.

Several accounting firms have set up new divisions to handle this for the mainstream side of the internet industry however, until just recently, the adult internet was unaware this was happening.

It would seem that although the international market place is a wholly viable one for adult webmasters to break into, it brings with it more complications that simply breaking the language barrier.

This new law will be a good test of the adult industry processors to see how they can handle the economic changes of the global market place and, more importantly, how they handle the new frontiers of international marketing on a global scale.

Article written by Lee

Article Writing For Publicity And Credibility

Date: 2011.02.21 | Category: Writing | Response: 0

If you’re looking for a powerful way to get free publicity and build your credibility at the same time, then writing articles may be your answer.

If you’ve been on the Internet for a while, you’ve probably subscribed to a few ezines including this one, hopefully. Many ezine publishers will include an article written by a guest author. At the end of the article are a few lines of text about the author referred to as bylines or resource box. These lines of text are basically just an advertisement for the writer. They usually contain a couple of lines about the author and a web address.

The writer gives the publisher permission to publish their article, free of charge, in exchange for the publisher including the author’s bylines.

By writing articles and allowing them to be published, your articles will have the potential to be viewed by millions of Internet users. They may be published by several ezines with subscriber bases of a few hundred to several thousand. In addition, they may be displayed in ezine archives or on high traffic websites.

Most ezine publishers prefer short articles between 500 and 750 words. Short “tip” articles of just a couple of paragraphs are also very popular. Articles should be formatted to 65 characters per line or less, including spaces, and written in short paragraph sections.

When you begin writing your article, avoid using your standard word processing programs, as they do not allow for proper formatting. Instead, use a text editor such as Notepad. It should already be installed on your desktop.

When you begin typing your article, use a hard carriage return (hit enter) when your text reaches 65 characters, including spaces, and leave a space between your paragraphs. This will enable publishers to easily copy and paste your article into their publication. By taking the time to properly format your article, you will increase your chance of being published significantly.

Most publishers receive many article submissions each week and only select a few to be published. Here are some basic guidelines to assist you in getting published:

(1) Make sure you follow the publishers’ submission guidelines. Articles submitted to publishers that don’t follow the submission guidelines will most likely be deleted.

(2) Make sure your article is properly formatted.
Publishers won’t take the time to format your article.
They’ll simply delete it and move on to the next article submission.

(3) Keep your bylines down to 6 lines or less.
Publishers will not publish articles that contain excessively long bylines.

(4) Select a descriptive title to intrigue your readers.
Use a powerful headline that demands attention and try to keep it all on one line.

(5) Use proper grammar and spelling.
Publishers will not take the time to edit your article. Make sure you read your article several times and use spell check.

(6) Avoid articles that are nothing more than a sales letter.
Publishers want quality content and will simply delete an article that is written like a sales letter.

(7) Avoid referring readers to an affiliate URL.
Articles containing affiliate links may make your article appear to be biased and untrustworthy.

(8) Write your articles with a sincere desire to teach and inform. Talk to your readers and share your expertise.

Once you’ve written your article, you’ll need to develop a list of publishers that may be interested in publishing it. The best way to accomplish this is to display your articles on your website. Place a subscription box on each of your article pages to enable your visitors to subscribe. This list should be used to send your new articles to your list of publishers.

Of course, you are already in one of the right places to start, we are always looking for industry related articles so, if you been given the motivation by reading this article to give it a try why not submit one to us for publication?

Writing and distributing free articles on the Internet will be one of the best promotional decisions you’ll ever make. Not only will it provide you with free publicity, but if your articles are good, you’ll become a trusted professional in your area of expertise.

Article written by Lee

Over The Rainbow

Date: 2011.02.24 | Category: General | Response: 0

So why is the gay community over the rainbow? As promised today I’ll finish up on the symbols that are prevalent in the gay community. Last week I mention the Pink Triangle and it’s political emergence. And technically, the rainbow flag has a similar political beginning.

I always love answering a question with another question (it is one of those annoying habits I have). So, when I am asked the “what does the rainbow flag stand for?” I look forward to the response when I ask, in turn, “what do YOU think it means?” Here are my top three favorite answers:

1 – “It’s colorful and gay people like colors.”

2 – “It’s a dance party thing”

3 – “Judy Garland sang Over the Rainbow you guys loved her a lot.” (This has to be my all time favorite.)

Well like any political symbol, it is a way to distinguish ourselves. But this time we look not to a place in history but we look at the world in general. No matter where you are, whether it is on 8^th street in NYC or Rue du Champe in Paris, there will be a gay person somewhere. Now, some may argue that not all cultures and societies will have homosexual present. And I say, “BULL”. They are just in the closet somewhere.

So in essence, gay men and women cross every racial and religious culture around the world. We come in all shapes, sizes and colors. So what better symbol to show that with than a rainbow, all colors coming together as one – as one community. No matter how culturally diverse our world is, the gay community is made up from a piece of each.

The rainbow flag and the uses of the rainbow, in the gay community, are diverse. If you see it flying outside on a restaurant wall, and it means it is a gay or gay friendly establishment. See a set of rainbow rings around a friend’s neck, it’s gay fashion accessories. And see a political march with rainbows on posters and flags, it’s a gay assembly.

Last week I insinuated that the pink triangle was not a symbol to over use in design. Well, in my opinion, the opposite is true with the rainbow and the rainbow flag. I am NOT saying run out and re do all your sites with rainbow backgrounds. No! But, I am saying that the nature of this political symbol is somewhat more flexible. We ourselves took the rainbow and made jewelry, publications, tee shirts and such. And just as the pink triangle, the rainbow will say that it’s ok to be here, you have friends inside or you have nothing to worry about here – be yourself, this place is for you.

So what have we learned? What some people think as silly symbols that gay people use, actually have an origin – and, a potent origin at that. I still ask, for your own good, that you use them wisely and not over do it. The more you throw symbols around willy-nilly, the more people will think you do not know what you are doing. And the more that the surfers thinks that, the less likely you are to make the sale and get your business to grow.

Article written by Gary-Alan

Everybody’s Going Gay

Date: 2011.02.22 | Category: General | Response: 0

The past few months in the online adult industry have seen some rather dramatic changes happen from Visa charging additional fees to accept their credit cards online, Paypal no longer processing adult and, several large companies going out of business. However, one thing that also seems to be changing for the better is the amount of webmasters looking at new niches to promote. Surprisingly, the one niche that many webmasters are looking at is that of the gay niche.

Where The Money’s At.

For years, the adult webmaster community has been split amongst themselves, those that are only focused on straight niches and, those that focus on gay niches however. As of late, more and more of those webmasters from the first group are looking to make additional income from the gay niche. With this inevitable surge in gay adult sites coming online, there are bound to be some knock-on effects, and those webmasters just starting to look at the gay niche, will return to what they know. However, for most, i think this is going to be a fruitful encounter with one of the industry less socially accepted markets. Many of the webmasters who have been concentrating solely on the straight market for their income are going to realize just how much money they have been losing out on for all these years. And with this realization, comes its own set of problems for the industry as a whole.

The Gay Market.

Whether you are a straight webmaster or a gay webmaster the one thing that will become apparent is that your sites are going to reflect you in both, how they look and, the marketing techniques you use. More importantly, the attitude you have towards your gay surfers will in fact show through. Many of the straight webmasters whom, perhaps are not as tolerant towards this niche will, almost certainly fail before they can really get going. Marketing gay adult sites is far different than marketing straight adult sites. You have to know the surfer, understand what they want and, more importantly, know the terminology in the gay market. Many times a post can be seen on various message boards belittling those webmasters who are openly gay – often accompanied by terms such as ‘fag’, ‘queer’, ‘homo’, etc along with with images being posted that, quite frankly, would turn most people’s stomach.

The gay niche is much more than just a single online market, it is a whole lifestyle unto itself. While this lifestyle is affluent (having an abundance of readily available money to spend) it will take a lot of work and perseverance to ensure you get your share of this market.

Starting Points.

As I mentioned above, the main thing many webmasters are going to have to learn are the terms of the gay market, what is a twink? A bear? A cub? What products work well on gay sites? Despite what most people think, you cant just throw up some banners with naked men on them and turn a profit. Match your content to a niche and then, match your advertising to your content, what use is it having bears on a site if, the paysite you are promoting features Teens ‘n’ Twinks?

Similarly, your text links, ‘Click Here For Hot Gay Action’ might get you a few clicks but, by targeting your surfers even more, you will see how some of the gay niches can really turn a profit. The trick is this, get the surfer to your site and actually give them something that you think will suit their desires. This may mean making a 10% commission on a sex toy sale as oppose to a $35 commission on a pay site membership. And so be it. Those 10%’ers start to add up after a few sales.

In Summary.

To summarize, the gay niche will make you money however, to make money you will need to invest something that, perhaps isn’t for everyone – a lot of time and understanding. Marketing the gay niche isn’t for everyone, we have seen this too many times. However for those who are, spending time adding a gay market portfolio to there straight, with a little time increase your overall income significantly.

Article written by Le

Domain Name Registrars – Saving Your Business

Date: 2011.02.22 | Category: WebDesign | Response: 0

The one thing that all webmasters rely on to make their money is something that all of us have, a domain name however, what would or should happen to your domain names when and if, your registrar files for bankruptcy or just disappears, surprisingly enough i do not recall this ever happening in the industry to date however, being prepared for the worst case scenario is always a good thing.

Prevention Is Better Than A Cure.

As the age old adage goes.

The first thing that you should do before even registering a domain name is to check out the background of the company you are considering using, you need to ask yourself a variety of questions about this company including the following:

1) Is the registrar ICANN accredited?

ICANN (Internet Corporation for Assigned Names and Numbers) is a non profit corporation which was formed to assume the responsibility over the IP and entire domain name structure as we know it. This in essence means that if your registrar is ICANN accredited then you at least know your domain name registration will be handled professionally and, should be reasonably secure so far as your registrar going out of business.

2) What are you paying for?

Many of the domain name registration companies packages vary greatly, with some companies you register a domain yourself, some companies register it on your behalf and, others will register your domain including some form of hosting etc, check with the company you are using to see what added services and support features they offer.

3) What is their transfer policy?

Some domain name registration companies have specific policies so far as transferring domains to other companies, registrars and, individuals go. Check with your registrar before you buy your domain to see what this process involves and, as always, check with one of the other registrars to see which registration company offers the best ‘bang for your buck’.

Saving Your Name – Help And Advice.

First and foremost, if you discover your domain name registrar has closed its doors you should immediately send an email to them and ensure you keep a copy of this email for your own records, ask them what is happening and, more importantly, what controls / access you still have available (if any) to manage your currently registered domains.

Usually, you will receive some form of communication within a few days from your registration company letting you know what is happening and how you can continue to use your domain name.

In most cases, when a domain name registrar company closes what you will find is that they will already be in the process of transferring the management of your domain to another registration company.

If however, you are still able to control the domain yourself then you should immediately seek out a new registrar with whom you can manage the domain / domains that you own. At this point you should contact them and ask if they have any fast track solution to transferring your domain to their company.

If All Else Fails.

If after everything else you are still at a loss as to how your domain name transfer or registration is being handled then you should contact ICANN ( http://www.icann.org ) directly. Ultimately it is their responsibility to ensure that once you have registered a domain name, you are able to access it as you would like, in most cases however, contacting ICANN is always the last move you should make and, only use this form of information should you exhaust all other methods mentioned above.

Article written by Lee

Robots.txt – Control The Robots That Crawl Your Sites

Date: 2011.02.24 | Category: Search Engine Optimization, WebDesign | Response: 0

By writing a structured text file you can indicate to robots that certain parts of your server are off-limits to some or all robots. It is best explained with an example:

# robots.txt file for general use on web servers.

User-agent: webcrawler
Disallow:

User-agent: googlebot
Disallow: /

User-agent: *
Disallow: /cgi-bin
Disallow: /logs
The first line, starting with ‘#’, specifies a comment.

The first paragraph specifies that the robot called ‘webcrawler’ has nothing disallowed: it may go anywhere.

The second paragraph indicates that the robot called ‘googlebot’ has all relative URLs starting with ‘/’ disallowed. Because all relative URL’s on a server start with ‘/’, this means the entire site is closed off.

The third paragraph indicates that all other robots should not visit URLs starting with /cgi-bin or /log. Note the ‘*’ is a special token, meaning “any other User-agent”; you cannot use wildcard patterns or regular expressions in either User-agent or Disallow lines.

Two common errors:

Wildcards are not supported: instead of ‘Disallow: /tmp/*’ just say ‘Disallow: /tmp’.
You shouldn’t put more than one path on a Disallow line (this may change in a future version of the spec)
Ultimately, without the use of robots.txt files on your servers/domains, you are risking a variety of potential problems including, unauthorized access to your cgi directory, unauthorized viewing of your site stats, possible spamming of the search engines by accidental crawling of doorway pages.

One distinct advantage however of having a robots.txt file on your server is that, quite simply, you will be able to tell when and where your site has been indexed or potentially indexed as, all robots will automatically call for the robots.txt file BEFORE any other page on your server so, as long as you keep an eye open for any calls of this file, you can see who is knocking at your site for indexing purposes.

Below is a robots.txt example that you can copy and paste into a text document to use on your own server:

<!–Start Copy Below This Line–>

User-agent: *
Disallow: /cgi-bin
Disallow: /logs

<!–End Copy Above This Line–>

The above will allow all spiders to crawl all of your site except the subdirectory’s ‘cgi-bin’ and ‘logs’ which, may be altered to suit any subdirectory’s you do not wish the spiders to crawl on your server.

Article written by Lee