-
Unspoken Secrets
here is a belief among new webmasters that there is a set of golden rules among those of us whom have been at this business for many years to making money well guess what, these are those unspoken secrets….
Secret #1.
There are no secrets to becoming an adult webmaster. One of the most common reasons why so many new webmasters give up in this business is because they hear about the money that the big guys make, submit two TGP galleries to The Hun, get a $200 hosting bill and no signups. They then figure that the affiliate they are using is rubbish and then spend the next 2 months surfing the Net looking for the “be all and end all” of affiliate programs which is going to net them $2 million within the next 6 months. The fact of the matter is, that genie in the lamp you are looking for doesn’t exist. It all comes down to hard work, perseverance and common sense this is a business venture you have started like it or not and, it needs to be treated as one.
Secret #2
Don’t give up. You need to treat what your doing as a business. Of course, you can have fun with it but, at the same time you need to be making continual business plans for your development in much the same way as if you were self employed. Which, of course as soon as you start to make some money you will be.
Secret #3
Stay focused. There are a million and one ways to make money in the industry. You could build free sites, TGP galleries, AVS sites, pay sites, search engine pages, the list goes on. There are many different avenues to drive traffic. The key is to try everything. Eventually you will find the one venue that your are very good at. Continue with this until you know this in your sleep – work it so often and well that you master it. One other thing to remember, it wont make you money overnight, if you think it will, your starting out for the wrong reasons, putting it simply, Adult web mastering isn’t for you.
Secret #4
Build it and, it (And they) will come, don’t spend time worrying about your keywords, meta tags, the amount of images you are using, just because someone tells you this is what is required of your sites. Just build a site or gallery submit it, forget about it and get started on building your next, and the next, and the next site after that. Just keep on moving forward, if you forgot to do something on one of your sites don’t worry, add it to the next site that you build.
Secret #5
Start to network. Other webmasters are great for bouncing new ideas off, a group of webmasters who start at the same time all communicating with each other are going to develop more than a sole webmaster on their own. Networking is a big part of any successful webmasters daily routine. Make the most of the people you meet on the boards or aim or icq, it will benefit you long term.
Secret #6
This one you will probably hear day in, day out for the rest of your business life. “It’s too late for newbies to make money” I am sorry to be the one to tell you this but, this is rubbish. As long as you follow the basic rules above and work at it, you can and will make money in this still thriving industry.
Now you have heard the “secrets” you have wanted to know its up to you to put them to good use.
Article written by Lee
-
Whats A Twink
I think you would be surprised at the number of times I have been asked, “What’s a Twink?” Or my, personal, favorite “So how do two men have sex?”
If you know me, you know I am never really bothered by questions – I never mind people’s candor. What does bother me, though, is the bulk of the people who ask are trying to work the gay market. Unsuccessfully, I might add.
So, let’s expand our webmaster knowledge. Don’t worry, I am not about to explain how two men have sex (that I will save for my story site LOL)
I have decided, however, to give you a Gay Glossary of sorts. There are many terms that can be included here. But, I thought to start off, we will hit the basics and add to it as time goes on. Keep an eye for future articles and additions on Gay Wide Webmasters.
Here is a list of the most common terms on the Gay Adult ‘net:
- Twink – A buff and lean young man; a 20 something. Age and leanness make the guy a twink. The best example is your typical bar room stripper.
- Teen – This is the same thing as in EVERY market, hot young and under twenty – but LEGAL at 18.
- Hunk/Stud – Beautiful, built and beefcake. The guys commonly posing for calendars and such.
- Chubs (or chubby) – Just what it sounds like, a larger man. Not just muscular weight, most often heavy set.
- Bears – This is a HAIRY man. Not a young guy, most often over thirty, sometimes with a husky build (but not always). You will probably see this niche sold as “Real Men”.
- Cub – This is a YOUNG hairy guy. Sort of a HAIRY TWINK…
- Daddies – Daddies are men over thirty, who like younger guys. (Just a side note here to clear up some misconceptions, not all gay men want to do young guys!)
- Sons – the young male counterparts to Daddies. The young of the two can be a twink, cub or teen – makes no difference. This term is based on the visual couple. For our non-gay friends think of this as an example: the businessman who goes away for the weekend with his NEICE. See the comparison? LOL
- Transvestite – is a man who dresses as a woman. Whether it is simple underwear or all our female attire. A lot of webmasters confuse this with Transsexual.
- Transsexual – is a person who has decided to make the complete change and have a sex change.
- Trans Gender – this is a more general term for people who live life as the opposite sex. They are in the process of, or have completed, a sex change.
So know go have some fun. Try to classify your friends and lovers in these terms! I enjoy it when a webmistress comes up to me and says, “I heard your interview and I think my husband is a bear. But he’s a little twink-ish. What do you think?” This just makes my day! I have to laugh at the look on their husband’s face when they think I’m going to ask to examine them or something. “Turn you head and cough – OH! You’re a Twink…”
Along with what works, I have to offer some terms that DO NOT work. If used incorrectly or out of context, then your attempt at marketing to gay men can be dreadful.
- Faggot – I hate this word, personally. This is a word that is as derogatory to the Gay Community as other words are to a race.
- Nancy boy – a typically non-US term that basically is used as a derogatory manner.
- Lil Boys – a derogatory term used primarily in the Southern and Mid-Western US. It’s derived from the negative connotation that gay men are pedophiles.
- Gay Owned and Operated. Allow me to be blunt – If it is not true, do not say it. This is probably the most OVER used phrase in the gay adult market. So much so, it is relatively meaningless today.
We all need to remember from time to time, the key to success in any business is knowledge. Whether you are up selling to a sponsor or designing sites, knowing some of the more common terminology in the gay community can only help your ventures in the gay adult market. And remember, do not be afraid to try and NEVER be afraid to ask.
As always, good luck!
Article written by Gary-Alan
-
Message Boards – Getting The Postings
There are plenty of free scripts available on the web which will enable you to start your own online discussion forum.
However, having the most sophisticated and advanced features in a forum does not mean it will be successful by any means. It is the people who post on the forums that make them good not the scripts used to maintain them. This article is dedicated to getting people posting on your forums.
It can be really quiet on your forum especially at the beginning when there are no real discussions going on, this is because people don’t want to be the first to post. They want to see some topics that interest them already being discussed, they want to be able to read through some posts first before having to post themselves. With this in mind, you should also be aware that just because a forum doesn’t seem busy, it can in fact get thousands of hits a day. Lurkers make message boards what they are, and, for this reason alone you should act professionally and courteously when making posts on any forums, one day, you might have to do business with one of these so-called ‘lurkers’ and, if they have seen you in a bad light on a forum, the chances are they wont want to know you. So what do you do to get your message board off the ground? There are a number of things actually and, some of these are as follows.
First of all, make sure the signup process is as simplistic as possible, even better, ensure if the user doesn’t want to register, they don’t have to! To attract people to post on your boards why not get a few of your friends together and start some discussions of your own, you could even register a couple of different usernames so that your forums look busier than they actually are thus cementing the thought that your forums are active and will have worthwhile discussions taking place.
To find some current topics for your forums why not visit some of the other message boards and see what people are discussing, if a topic is receiving a lot of interest, post about it on your forums. This is especially a good idea if your forums are specific to one area after all, if your competitions forums posters are interested in this topic, your forums users will be too.
Another great way to get people to start talking on your discussion forums is to use a contest. For example, currently on European Webmasters we have a content giveaway whereby for each post you make you gain a point and, once you have reached a certain level of points you will be able to download exclusive content sets. If you are going to start a competition then make sure people are aware of it, post some information about it online and get something to giveaway that everyone wants.
One other method to get additional posts is to put up subject lines and links to the topic threads from your main website page. Again, if you publish articles on your site create a link to your discussion forums and invite people to openly discuss the articles in question, not only will you get people posting on your message board but, you will also gain invaluable information from your user as to what they look for and want in articles you write.
If you publish a newsletter, again, this is another great way to build interest in your message board. Simply ask them to post, tell them you are building up a great resource and, you need their help as much as you need theirs to make this happen. If you have provided them with high quality information in your newsletter the chances are they will repay you by posting on your message board.
In conclusion, you should basically try and plug your message board wherever you can by doing the above, create banners for it and display them on your website, place a note in your signature file encouraging people to visit and post if they require assistance with anything. Its hard work but, once you have a few regular posters it will take off.
Article Written By Lee
-
Over The Rainbow
So why is the gay community over the rainbow? As promised today I’ll finish up on the symbols that are prevalent in the gay community. Last week I mention the Pink Triangle and it’s political emergence. And technically, the rainbow flag has a similar political beginning.
I always love answering a question with another question (it is one of those annoying habits I have). So, when I am asked the “what does the rainbow flag stand for?” I look forward to the response when I ask, in turn, “what do YOU think it means?” Here are my top three favorite answers:
1 – “It’s colorful and gay people like colors.”
2 – “It’s a dance party thing”
3 – “Judy Garland sang Over the Rainbow you guys loved her a lot.” (This has to be my all time favorite.)
Well like any political symbol, it is a way to distinguish ourselves. But this time we look not to a place in history but we look at the world in general. No matter where you are, whether it is on 8th street in NYC or Rue du Champe in Paris, there will be a gay person somewhere. Now, some may argue that not all cultures and societies will have homosexual present. And I say, “BULL”. They are just in the closet somewhere.
So in essence, gay men and women cross every racial and religious culture around the world. We come in all shapes, sizes and colors. So what better symbol to show that with than a rainbow, all colors coming together as one – as one community. No matter how culturally diverse our world is, the gay community is made up from a piece of each.
The rainbow flag and the uses of the rainbow, in the gay community, are diverse. If you see it flying outside on a restaurant wall, and it means it is a gay or gay friendly establishment. See a set of rainbow rings around a friend’s neck, it’s gay fashion accessories. And see a political march with rainbows on posters and flags, it’s a gay assembly.
Last week I insinuated that the pink triangle was not a symbol to over use in design. Well, in my opinion, the opposite is true with the rainbow and the rainbow flag. I am NOT saying run out and re do all your sites with rainbow backgrounds. No! But, I am saying that the nature of this political symbol is somewhat more flexible. We ourselves took the rainbow and made jewelry, publications, tee shirts and such. And just as the pink triangle, the rainbow will say that it’s ok to be here, you have friends inside or you have nothing to worry about here – be yourself, this place is for you.
So what have we learned? What some people think as silly symbols that gay people use, actually have an origin – and, a potent origin at that. I still ask, for your own good, that you use them wisely and not over do it. The more you throw symbols around willy-nilly, the more people will think you do not know what you are doing. And the more that the surfers thinks that, the less likely you are to make the sale and get your business to grow.
Article written by Gary-Alan
-
JavaScript – Redirecting Foreign Surfers
At some point or another we are no doubt going to have the need to redirect some or all of our surfers based on the language they speak, this snippet of JavaScript when placed on your page will enable you to do just that without the need for .php or other more complex scripting.
Here is the coding that you need to place between your <head> and </head> tags:
<SCRIPT LANGUAGE=”JavaScript1.2″>
<!– Begin
if (navigator.appName == ‘Netscape’)
var language = navigator.language;
else
var language = navigator.browserLanguage;if (language.indexOf(‘en’) > -1) document.location.href = ‘english.shtml';
else if (language.indexOf(‘nl’) > -1) document.location.href = ‘dutch.shtml';
else if (language.indexOf(‘fr’) > -1) document.location.href = ‘french.shtml';
else if (language.indexOf(‘de’) > -1) document.location.href = ‘german.shtml';
else if (language.indexOf(‘ja’) > -1) document.location.href = ‘japanese.shtml';
else if (language.indexOf(‘it’) > -1) document.location.href = ‘italian.shtml';
else if (language.indexOf(‘pt’) > -1) document.location.href = ‘portuguese.shtml';
else if (language.indexOf(‘es’) > -1) document.location.href = ‘Spanish.shtml';
else if (language.indexOf(‘sv’) > -1) document.location.href = ‘swedish.shtml';
else if (language.indexOf(‘zh’) > -1) document.location.href = ‘chinese.shtml';
else
document.location.href = ‘english.shtml';
// End –>
</script>To add additional language redirects to this JavaScript all you need to do is duplicate the:
else if (language.indexOf(‘zh’) > -1) document.location.href = ‘chinese.shtml';
Section of the coding changing the (‘zh’) language code to that of the language you wish to redirect.
Article written by Lee
-
Fighting Back Against Computer Viruses
Only 10 years ago, the only way that a virus could have infected your computer was through you physically putting a floppy disk into your hard drive and running the program file that initiated the virus however, today’s viruses have become more complex in how they operate and, can spread like wild fire if they are not recognized as a virus doing damage not only to your computer but that of your associates and peers.
So how do we minimize the risk of virus infection on our computers which, lets face it, for the Adult Webmaster are our livelihood and, need to be protected at all cost. This is what we will take a brief look at in this article, how to minimize the risk of getting a computer virus and, heaven forbid, how to deal with a virus if you actually have one on your machine already.
Computer Viruses – A Little History.
1981 – The first widely accepted computer virus is recognized. The ‘Elk Cloner’ virus was spread through floppy disks and, whilst by today’s standards, it was not a high risk virus it, non the less was still a virus, in fact, all this first virus did was to display a message on your computer screen.1991 – The ‘Tequila’ virus was the first recognized ‘polymorphic virus’ meaning, it was the first virus which actually altered its state once on your machine in order to remain undetected.
2001 – 2001 saw the spawn of several highly potent ‘worm’ orientated viruses from the ‘Gnuman’ virus, the ‘LogoLogic Worm’ and the famed (is that the right word?) ‘Nimda worm’. These ‘worms’ were the first of their kind to actually spread themselves across the World Wide Web through infecting users email clients (specifically Outlook Express) and sending itself to every email address the infected machine had in the address book.
Virus Infections – Prevention Better Than Cure.
So we now have a little background on the main viruses over the last 20 years we will take a look at the ways in which we can prevent viral infections on our computers after all, as the subtitle for this paragraph says, a prevention is far better than having to find a cure.Here are a few simple ways you can help to prevent the spread and infection of computer viruses.
1) Install a good anti-virus software package on ALL of your computers, Norton and, Mcafee, both have some good products to aid you in preventing viral infections of your PC. Also, you should ensure the software package you do install has updated virus definitions, a new virus is created almost every week so you need to ensure your software keeps up-to-date on the latest infections.
2) Scan any and all email attachments you receive, irrespective of whether you know the sender or not, anything that you get sent in an email should be scanned, as mentioned above, there are complex viruses that can ‘send’ themselves from other peoples email address book, what happens if you are on the address book of an infected users computer? Can you afford not to be online?
3) Ensure that, once you have an Anti virus program installed on your machine that you have it setup to start each and every time that your computer boots, the first thing that should be loading on your machine is an anti-virus program, you only have one chance to prevent infection.
4) Avoid downloading .exe and .com files to your desktop unless, you know the person you are downloading the files from and, even then, before you launch or open the files ensure you scan them independently.
Virus Prevention – Overview.
Hopefully this article will have given you some insight into how you can protect both your computer and the computers of your associates and peers from computer viruses and, hopefully, by following the guidelines within this article your time spent online will be productive and virus free. One thing you should remember however is that no matter how hard we all try to remain virus free, someone will create a new program that ultimately, could infect our machines no matter how hard we try.Article written by Lee
-
Redirecting Questionable Adult Traffic
The one thing almost all reputable adult webmasters agree on is that one way or the other, we want to rid the net of those webmasters that profit from traffic primarily gained from either the direct promotion of or, targeting of, keywords relating to child pornography however, how can you tell what traffic you are being sent and, more importantly, how can you do something about the type of traffic that you receive to your site in order to filter out this unwanted traffic? The answer is simple, use a script to redirect the traffic elsewhere before it even hits your site.
Child Porn Redirection Php Script.
The following php script when used on your server will enable you to send unwanted traffic gained by the promotion of using ‘illegal’ keywords in the search engines and sites which link to your own.
<!– Start Copy Here –>
<?
// Redirect “Lolita” traffic
$refer_full_path = “$HTTP_REFERER”.”$PATH_INFO”;
if(( preg_match(“/lolita/i”, $refer_full_path)) ||
( preg_match(“/child/i”, $refer_full_path)) ||
( preg_match(“/preteen/i”, $refer_full_path)) ||
( preg_match(“/pre-teen/i”, $refer_full_path)) ||
( preg_match(“/pedo/i”, $refer_full_path)) ||
( preg_match(“/underage/i”, $refer_full_path)) ||
( preg_match(“/beast/i”, $refer_full_path)) ||
( preg_match(“/rape/i”, $refer_full_path)) ||
( preg_match(“/kinder/i”, $refer_full_path)) ||
( preg_match(“/incest/i”, $refer_full_path)) ||
( preg_match(“/kiddie/i”, $refer_full_path))) {header(“Location: $refer_full_path”);
exit;
}?>
<?
$words=array(“childporn”,”underage”,”beast”,”interracial”,”lolita”,”preteen”);
for($i=0;$i<count($words);$i++){
if(eregi($words[$i],$HTTP_REFERER)){
header(“Location: http://www.fbi.gov/?CHILD_PORN_ON_DISK_LOGGED_AND_REPORTED”);
}
}
?><!– End Copy Here –>
In order to use this script, simply add additional keywords or partial word matches to the top part of the script and, include the bottom half of the script at the top of your HTML coding.
Any traffic being sent to your site via keywords which you have specifically told the script not to allow access to your web site will be forwarded to the url in the bottom part of the php script which again, can be changed to wherever you like.
Article written by Lee
-
Utilizing All The Content You Use On TGP Galleries!
Utilizing All The Content You Use On TGP Galleries!
For the last few months i have been experimenting with a few new (but old, if ya see what i mean) ideas.Basically i have adapted Voltars 3x3x3 project slightly and incorporated it into the TGP/TGP2 game.
The results have given me some of the best sales ratios ever! We’re talking under 1:60 almost consistently across all sponsors!!
What you did is thus….
Design a NICE 15 pic tgp gallery in a table preferably four rows high and 5 columns across.
In the first row place one thumb in each cell (5 pics).
In the second row place one odd sized banner in the first cell and merge cells 2,3 , 4 and, 5.
In the cells you have merged place some descriptive text about the pics you are using in under two paragraphs and include a line of text where you MENTION the sponsors site name.
In row 3 place one thumb in each cell (5 pics).
in row 4 place on thumb in each cell (5 pics).
under the table place a juicy text link.
below this juicy text link leave a line or two and add a money bar (one row by three columns using a contrasting color to the background of your gallery page).
ALL pics go on html pages using the same layout as your main TGP page but, without the table included.
Repeat this process 4 times once for four different niches, saving each niche in their own sub folder for the main site i.e:url/sub folder/niche 1
url/sub folder/niche 2
url/sub folder/niche 3
url/sub folder/niche 4You should now have 4 TGP galleries using 15 pics on each in four niches (60 pics in total).
At the top of each of these main pages place a three cell one row table.
in this table use text links to describe each niche i.e:
Teen
Gay
Lesbian
Maturein the template for the teen niche you use text for the gay, lesbian and, mature templates and link these to the appropriate TGP template.
This is where the fun begins…..
Take all four TGP niche galleries you have just made and do the following….
Delete the first row of cells (including pics) on all TGP galleries and save the new page layout in another folder using the same navigation structure as the original ie:
url/sub folder 2/niche1
url/sub folder 2/niche2
etc
etcYou should now have the following:
four TGP templates consisting of 15 pics each.
four TGP templates consisting of 10 pics each.Again, link these new 10 pic templates to each other in the same way as you did the 15 pic templates.
Repeat this procedure to make the 10 pic TGP page using the original TGP page BUT, this time, delete the bottom row of cells.
You now have the following:
4 niche 15 pic TGP pages
4 niche 10 pic TGP pages
ANOTHER 4 niche 10 pic TGP pagesStill with me?
Its gets better….
Take your original 15 pic TGP template and do the following…
Delete the first and the third row of cells including all the pics and save them in ANOTHER folder using the same structure as before i.e:
url/sub folder 3/niche 1
url/ sub folder 3/niche 2
etc
etcAgain link these in the same way as before using the text links at the top of the pages.
What do you have now?
4 TGP pages using 15 pics
8 TGP pages using 10 pics
4 TGP2 pages using 5 picsCan you guess what’s next?
Take that original template again and…
Remove the first and fourth row of cells do the same when you save it as you have done above…
What do you have?
4 TGP pages using 15 pics
8 TGP pages using 10 pics
8 TGP2 pages using 5 picsDo it once more using the original templates but removing the final set of 2 rows of cells you now have…
4 TGP pages using 15 pics
8 TGP pages using 10 pics
16 TGP2 pages using 5 picsIn the root of the domain where you are going to upload these pages do the following…
Create a simplistic warning page… link this to a multi site FPA… this multi site FPA links to each of the niche (15 pic) TGP templates you have made.
On this same fpa place an email erotica collection box, banner exchange code and a toplist counter code along with an exit console going to a MC POTD program.
Now, what you need to do is start submitting these pages to the TGP/TGP2’s submit one a day or as many as you can until you have submitted them all.
Ok still here?
Next you have to duplicate ALL of the above again.
Place the duplicate of the above into a separate set of folders/sub domains etc.
On the new root you have created, link to another multi site FPA AND the first root index.html page.
You now have the makings of an EXCLUSIVE 3x3x3 TGP hub which, you can leave alone and submit all of the pages intermittently to any TGP/TGP2 you wish for LIFE!!
All the time you are adding new sections to it and building your link pop in the search engines because you are continually interlinking to each of the separate niche and relevant sub niche folders.
Of course, you NEED to get the meta’s sorted on the first main template for this to work but, if you change all subsequent meta descriptions etc you’ll have a very reasonable listing in Google for the relevant search terms
Article written by Lee
-
Adult Webmaster Conventions – Remember When?
Most webmasters who have been working in the adult industry for a year or two have attended an adult webmaster convention of some sort and, those of us who had the privilege of attending our first webmaster show three to four years ago will know one thing that many others don’t, there used to be a time when you could attend a webmaster show for example, the one held at Disney World (how many of you remember that?) when apart from being relaxing, you would also get a lot of business done alas, times have changed.
Fighting, Drug Taking, Getting Drunk, Rape Accusations, Theft and Criminal damage would be the headline in the national press should one of these reporters attend today’s webmaster convention, instead of a relaxing atmosphere in which you can get business done you find a slew of webmasters who, well, lets be honest here, are only attending these shows for one reason, to get drunk (for free) and to have a laugh.
Of late there seems to have been an influx of people (notice I used the word people and not ‘adult webmasters’) joining the adult industry who would appear to think that our business model is one of all night partying, sordid sex sessions and drug taking, whilst, I will be the first to admit this has gone on in the past it would certainly seem that the last 2-3 years has seen more than its fair share of the negative events happening at these conventions.
Take for example the recent Hollywood Internext Expo show, one incident that comes to mind was the fool who decided it would be a good idea to smash the mezzanine canopy above the hotel lobby, to my knowledge the culprit was never found however, I am sure there are those out there who knows who it was and well, lets be honest, they quite possibly you could be reading this article.
So what happened? How did these shows change so drastically? Quite simply I think it was the sudden onslaught of press coverage about how the adult industry makes so much money for everyone and, whilst there is a lot of money to be made in the industry, it takes a lot of work to achieve making the reported figures by the press.
To many people are entering the adult industry thinking that the ‘norm’ is that of what you can find on your average day of visiting GFY and whist, this is perhaps true for the circle of webmasters who do frequent boards such as GFY for the vast majority of hard working webmasters in the industry, the behavior of those webmasters couldn’t be further from the truth.
So where is all the business being done at the shows? It would appear that event after event the open invite list of parties are being cut down to be taken over by more intimate and sociable private parties in fact, as a company we our ourselves holding several smaller invite only parties for those people that we actually want to do business with on both a personal and a professional level.
So what does the future have in hold for the adult industry events, in all honesty I do not know however, one thing is for certain, if things continue on their current route, things are going to become a whole lot worse before they get any better.
Article written by Lee.
-
Robots.txt – Control The Robots That Crawl Your Sites
By writing a structured text file you can indicate to robots that certain parts of your server are off-limits to some or all robots. It is best explained with an example:
# robots.txt file for general use on web servers.
User-agent: webcrawler
Disallow:User-agent: googlebot
Disallow: /User-agent: *
Disallow: /cgi-bin
Disallow: /logs
The first line, starting with ‘#’, specifies a comment.The first paragraph specifies that the robot called ‘webcrawler’ has nothing disallowed: it may go anywhere.
The second paragraph indicates that the robot called ‘googlebot’ has all relative URLs starting with ‘/’ disallowed. Because all relative URL’s on a server start with ‘/’, this means the entire site is closed off.
The third paragraph indicates that all other robots should not visit URLs starting with /cgi-bin or /log. Note the ‘*’ is a special token, meaning “any other User-agent”; you cannot use wildcard patterns or regular expressions in either User-agent or Disallow lines.
Two common errors:
Wildcards are not supported: instead of ‘Disallow: /tmp/*’ just say ‘Disallow: /tmp’.
You shouldn’t put more than one path on a Disallow line (this may change in a future version of the spec)
Ultimately, without the use of robots.txt files on your servers/domains, you are risking a variety of potential problems including, unauthorized access to your cgi directory, unauthorized viewing of your site stats, possible spamming of the search engines by accidental crawling of doorway pages.One distinct advantage however of having a robots.txt file on your server is that, quite simply, you will be able to tell when and where your site has been indexed or potentially indexed as, all robots will automatically call for the robots.txt file BEFORE any other page on your server so, as long as you keep an eye open for any calls of this file, you can see who is knocking at your site for indexing purposes.
Below is a robots.txt example that you can copy and paste into a text document to use on your own server:
<!–Start Copy Below This Line–>
User-agent: *
Disallow: /cgi-bin
Disallow: /logs<!–End Copy Above This Line–>
The above will allow all spiders to crawl all of your site except the subdirectory’s ‘cgi-bin’ and ‘logs’ which, may be altered to suit any subdirectory’s you do not wish the spiders to crawl on your server.
Article written by Lee
Premium Sponsors
Categories
- 2257
- Billing Solutions
- Blogging
- Branding
- Content
- Domain Names
- Employment
- Forms & Contracts
- General
- Hosting
- Link Lists
- Opt-in Mail
- Paid Traffic
- Pic Posts
- Promotion
- Scripts
- Search Engine Optimization
- Sponsors
- TGP
- Traffic
- Tutorials
- Viral Marketing
- WebDesign
- Writing