• Newsletter Know How – A Guide To Writing Newsletters

    Date: 2011.02.23 | Category: Promotion, Writing | Response: 0

    Email newsletters are a great way to make contact with existing clients and, prospective clients which makes them a great marketing tool. The biggest bonus however, over Email newsletters rather than print ones is that you have virtually no cost involved with writing and, ultimately, mailing them out.

    There is a wide variety of online newsletter on the internet. however, probably the simplest form for most webmasters is the email newsletter. Email provides you with a quick way to keep your name out there, build trust with you readers and, hopefully attract a few sales. If you archive your newsletter also this is a great way to get additional pages into the search engines and, as such, it then becomes a good traffic driver to your website regardless of the amount of subscribers you have. However one of the most important benefits of having an email newsletter sent on a regular basis is that it build brand recognition of your website or business.

    Your newsletter should provide information, not just of an advertising nature but that, should also be informative to your subscribers. In doing this you are offering information of value and, ultimately, ensuring that your subscribers remain. However, remember newsletters are for promoting your area of expertise not just a vehicle for revenue generation.

    One of the first things you should consider when deciding to do a newsletter is the amount of commitment you will have to put towards the project. Whether you start a monthly, weekly or, daily newsletter a great deal of commitment must be made to both the newsletter and, more importantly, your subscribers. For example, We generally spend a good 3 days researching the articles that have been submitted and trying to ensure that the details contained within them are relevant at the time of sending the email, this is not an easy task especially when the adult internet is constantly developing. If you don’t have the time to research and maintain consistency in your newsletter, then don’t even consider doing one, it isn’t for you at the present time.

    Now you have decided you can make the commitment required to run a successful email newsletter what do you do next? Collect email addresses. Probably one of the easiest ways to get email subscribers to join your list is to go through your current mail clients contact list and manually add those email addresses or, further still, you may even have an opt in collection program that you have been using for a while but have never really had the time to commit to writing a newsletter worthy of sending out. Remember though, an email list is only as good as the addresses contained within it. You want to run an OPT-IN email list, any other type of list will be considered Spam and, that will do you more damage than good. Of course, if you haven’t already got an email collection program you can find one online there are literally hundreds of them floating around that you can get for free from sites like http://www.hotscripts.com.

    So, you now have your emails, onto writing your newsletter. I think the greatest part of email is that it is somewhat less formal than writing for print copy. Email in fact tends to be very informal in comparison. The best way at approaching writing for an email newsletter is to write how you talk, it is more enjoyable for the reader and gives your newsletter a personality that no one else can emulate. Content wise, that depends on what you do, what would YOU like to see if you subscribed to an email list of the type you are going to be running and, more importantly, what would ensure you stay subscribed to the newsletter? Ask your friends, colleagues and, peers what they would like to see in an email newsletter, often they have been looking for information that is not readily available, this is an area where you then start to provide a service that no one else does.

    So you are at the point where you are starting to realize that running a newsletter is a lot of work correct? Well, it is. However, the effort, work and, time spent in putting your newsletter together will help you build a relationship with people whom, in all actuality, you may probably not ever meet in a face to face situation and, more importantly, it can become a great income generator over time.

    Article written by Lee

  • Making Money From Adult Webmaster Referrals

    Date: 2011.02.21 | Category: General | Response: 0

    One method of income generation that is very rarely talked about in the industry is that of the webmaster, from the new webmasters to the adult industry right up to those who have been in the biz some time.

    One of the easiest ways to generate ‘long term’ income is by referring webmasters to programs using your own linking (referral) codes.

    However, this is for some a huge step out of their everyday site building even though it shouldn’t be.

    This is what we will investigate in this article.

    By now we should at least have a fair few sites online and getting surfer traffic but, how many of us get traffic from other webmasters and don’t even know about it?

    The chances are, if you submit a site to ANYWHERE you will at least get one webmaster hit to it regardless of whether it is a webmaster of a TGP, Links List, Dmoz Editor etc these are all potential money makers for you.

    One of the easiest ways to make money from other webmasters is to simply have a link on each and every site you build in text that says ‘webmasters’ this should be linked to a single page on your domain with a listing of various affiliate programs you use or can recommend.

    Often placing a button or a banner on this page for each of the referral programs will work much better for you as the webmaster can actually see a little more information about the sponsor you are trying to make them join.

    We all know of some of the big sponsors like ARS, Topbucks, Sic Cash etc etc but, what about the smaller programs that, perhaps not many webmasters know about, these would be the ideal types of sponsor to use on your webmaster page.

    However, in addition to the affiliate programs you could also recommend hosting companies, content providers etc etc for example, the following companies have referral programs that you could use on this page:

    Content Providers:

    http://content.xphotography.com
    http://matrixcontent.com
    http://pixmasters.com

    Hosting Providers:

    http://webair.com

    Message Forums:

    http://arsresources.com/forum/
    http://xnations.com

    Traffic Generation:

    http://adultbuttonexchange.com
    http://pornclient.com

    As you can see from the small list above, you could have more than one way for webmasters to make you some additional profits in the form of either, cold hard cash or, from some of the places, additional traffic to your sites.

    Hopefully list brief article will have given you some ideas as to how you can make some additional money from webmasters visiting your site and, don’t forget, once you have the page linked on your site, it will, inevitably get into the search engines, so keyword the page up and you might start generating more webmaster traffic than you could imagine.

    Article written by Lee.

  • Building A Surfer Trap – Stage 2

    Date: 2011.02.21 | Category: Traffic | Response: 0

    In the last tutorial we hopefully got the foundations of our surfer trap laid and in this stage, we will start to put this thing together.

    Ok the first step of stage two of building your surfer trap is to start linking each of the individual FPA’s to your Multi-Site FPA.

    The easiest way that I have found to do this is to give each FPA its own sub directory on your server and have the html page named index for each of the single site FPA’s.

    So for instance, if you have the site All Petite on your Multi-Site FPA you would link it to:

    mydomain.com/all-petite/

    Or whatever you called the sub directory for the All Petite single FPA.

    Now, once you have linked these single FPA’s to your Multi-Site FPA we need to start ‘playing’ with them once again.

    You should now have copy’s of your single site FPA’s and Multi-Site FPA on both your server and Hard Drive.

    Take the copy’s you have on your hard drive and add a small NICHE pop up console to each of the single site FPA’s AND the Multi-Site FPA.

    I would suggest making 6 NICHE consoles.

    These consoles should be pure text and nothing else.

    Each of the links on this small console should link to a different niche of your single site FPA’s I usually go with one link for each of the following niches…

    Gay
    Mature
    Teen
    Fetish
    Hardcore
    Asian

    Plus, I usually add a link at the bottom of my console which goes to the POTD program.

    Once you have these consoles built you should upload them to their OWN sub directory on your server, I would suggest calling this directory ‘consoles’ and calling each of the niche consoles the name of the NICHE they represent.

    So, you should now have the following on your HD and on your server:

    1 Multi-Site FPA
    50 or so Single Site FPA’s (All Linked From The Multi-Site FPA)
    6 Small Pop-Up Consoles (Popping Only One On The Multi-Site FPA and The Single Site FPA’s, each different niche Single Site FPA pops a different niche console however.)

    You now have to check that your surfer trap is working so far.

    Article written by Lee.

  • The Gay Adult Market – Gay Subcultures

    Date: 2011.02.24 | Category: Promotion | Response: 0

    I thought I would take a break from writing about online adult marketing for a change and instead, give many of the straight webmasters looking to break into the gay market a bit of an insight into what makes a gay man ‘tick’ and, whilst this is going to be different from guy to guy, the one thing that will remain constant throughout is that this article is based on the gay lifestyle and subculture in both society and, the internet.

    What Is A Subculture?

    Lets start at the beginning shall we? To understand this article you are first of all going to need to understand what a subculture is well, a subculture as defined by http://www.dictionary.com is, ‘A cultural subgroup differentiated by status, ethnic background, residence, religion, or other factors that functionally unify the group and act collectively on each member.’

    In layman’s terms, that means, that a subculture is a group of people (or individual) that have something different about them than other people.

    In the instance of this article we are going to be looking at the gay subculture and specifically, how this culture is broken down into its own set of subcultures.

    The Gay Subculture – Lets Define The Gay Culture.

    Gay culture is the heritage of culture, knowledge, and references to which gay people fall heir by the fact of their sexual orientation.

    Some individuals argue that there are to many gay people who do not participate in ‘gay culture’ for the concept to be meaningful or that gay culture in itself is placing labels on the stereotype most people think of when they thing about gay people however with this said, there are also those who argue the flip side of the coin. One thing is for sure however, gay culture certainly does exist and, within this culture itself there are many complex facets ranging from sexual preferences through to the types of vacation people take.

    Gay Subcultures – The Facets.

    Whether we choose to think about the different subcultures within the gay community or not one thing is for certain, the subcultures in our community are wide and far reaching from your local gay bar to Soho in a city thousands of miles away most of the gay subcultures can be found almost anywhere in the world and, from this aspect, it makes gay individual no different than other person. For example, the BDSM lifestyle is large in the straight culture however, it is also hugely popular in the gay culture to often with groups of individuals who devote their whole way of life to this. In essence however, many of the gay subcultures are no different to that of their straight counterparts the only difference being, the individuals who practice them, are gay.

    Gay Subcultures – An Overview.

    For many webmasters who currently work in the straight market their first attempt and even their second attempt at marketing to the gay lifestyle may fail however, many do not realize the simplest of rules when it comes to marketing products and services to gay individuals online, you need to be as, if not more, detailed and direct to gay surfers than straight, give them something they can find anywhere at anytime and the chances are they will not buy however, give them something that suits their tastes perfectly and you have a customer for life.

    Article written by Lee

  • Writing A Business Plan – The First Step To Success

    Date: 2011.02.21 | Category: Forms & Contracts, Writing | Response: 0

    Regardless of what type of business you want to run be it a bakery, a funeral home or an online marketing empire the one thing that you will require is a business plan however, having never taken any business basics 101 classes what do you need to have and, what is required in your business plan?

    That is what we will touch on in this brief article from the basics to the not-so-basics by using a few simple steps you can be sure your business plan will contain all the information you need to make it up the ladder of success.

    Information Summary.

    As the name suggest this is a simple summary of what you are hoping to achieve from the name of your business, the goals, the concept and your plans. This should actually be no longer than two pages so that you can fill the needed information out clearly and concisely.

    Business Details.

    This will be the founding information of your business again you need to mention your business name, what type of business you will be running (Sole Trader, Partnership, Corporation, etc) in addition you will also need to include details of who the company owners are, whether it be yourself alone or a list of your business partners, this information will be invaluable to you at a later date you also need to include a brief history of the business whether you have been running it already for a number of days, weeks, month or, years.

    Products And Services.

    Basically, you need to write what exactly it is you will be selling. Are you going to be selling baked goods, marketing services or custom solutions. Keep this section simple, if you need to go into depth then do so at the end of your business plan by way of an appendices. Once you have listed several of the products or services you will be selling you should also list the features and benefits (Marketing points) of the products and services in addition, why do you think your business idea will succeed, what do you offer your clients that no-one else currently does? In addition you also need to have a basic pricing structure planned out, this will assist you in the future as, from your initial plans you can see what, if anything needs to be changed.

    Marketing.

    A business is more than just an idea. You need to have good pricing, affordable suppliers and a great marketing plan. This is what is also needed in your overall business plan. You should include details of your competitors products and how they compare to your own or, if applicable why your product will be a benefit to the market place should it not already be readily available. In addition the marketing section of your business plan should include a list of advantages as well as some of the pitfalls you are likely to see. You also need to include a section detailing your specific marketing strategies regarding your overall and individual products and services.

    Industry Economics.

    This area should detail some of the more specific areas of the industry you are going to be working in for example, what type of industry are you entering, how does the future for this industry look and, more importantly, how many existing companies work this industry as well as how do you successfully launch yourself into the industry. Parts of this area you may have covered in the marketing and previous section non the less, go over them again here in more detail.

    Target Market.

    We have detailed the business ideas and products and services but, what about your potential clients? What does your product offer then that they need which no other company does? What is your target client base? How will you reach your customers? All of this and more needs to be addressed in this area of your business plan.

    Competing Companies.

    What do your competitors do and, more importantly, who are they? Make a list of your potential competitors and address their weaknesses and strengths ensure that you leave no stone unturned when getting this information together often, you will realize that there are things your competition is doing that you can do much better then they.

    Don’t forget to address their marketing, what are they doing right and, more importantly to you, what are they doing wrong? How can you capitalize on this and make their errors benefit your long term business plans.

    Location, Location, Location.

    Location is everything and, this is particularly true in business for example, will you be working from an office, from your home or even your garage, do you require any special permits to operate your business? Can your customers reach you easily? All of these items and more need to be considered when writing this area of your business plan. By addressing any and all of these items you will find yourself with a much clearer picture of what is needed of you and your business.

    Stock + Supplies.

    Often overlooked in business plans the one thing you NEED to have is a list of potential suppliers and sub-contractors who can provide you with your products and services. What materials (if any) do you need? How much will these cost you? Where will you find your stock and supplies? By getting this information available now you will have a better idea once your business plan is laid out as to what you are able to offer easily and also it will enable you to lay your pricing structure out more concisely.

    Labor.

    So you have this great business idea all laid out but, who will be running this for you? Do you need additional staff or are you going it alone? Do you need to setup accountants, lawyers, insurance companies, consultants, bank accounts etc etc? All of these aspects of your business need to be laid out here.

    Capital.

    Money, as they say, makes the world go round. Your business is no different, how much financial outlay do you require to get your business off the ground? Do you need to ask for a loan from your banking facility to help you out? How much income are you planning on making within the next 6 to 12 months? How much of this income needs to be invested back into the business? All of these ‘financial’ question need to be answered in this area of your business plan only then will you begin to see the bigger picture of your business idea.

    Hopefully this article has given you some solid advice and information regarding planning for your new business and, once you have all of the above laid out you should be able to see whether you will succeed or fail in your new idea.

    Also, by planning now for the future of your business you should hopefully be able to avoid any hidden costs which can often push a company to the edge.

    Article written by Lee

  • Pay Per Click Or Search Engine Optimization

    Date: 2011.02.24 | Category: Paid Traffic, Search Engine Optimization | Response: 0

    What would you choose to run your business? Well each has their own benefits and drawbacks over one another.

    PPC or SEO The Breakdown.

    Pay Per Click or, PPC as it is most often referred to looks to the novice to be the better option for ‘immediate’ traffic results, you enter your desired keywords, place your minimum / maximum bid amount and you are set for top PPC engine listings for as long as you can maintain the balance in your engine account.

    Search Engine Optimization or, SEO on the other hand, is the more traditional way of attaining high ranking search engine pages. Either you or an SEO expert optimizes your websites pages and random other elements of your website and hopefully within a month or two, you achieve high rankings in the major search engines.

    So Which One? PPC or SEO?

    Generally speaking, SEO work is most commonly more cost effective to your business than utilizing PPC results to gain your traffic, you could pay an SEO expert anywhere from $500 plus to optimize your site and get high rankings indefinitely or, you could put that $500 into a PPC engine account and get high rankings until such time as your account balance runs dry.

    But lets look at this in terms of actual traffic…

    Say you get 1000 visitors to your SEO based website which you paid $500 for, each visitor has cost you $0.50c now lets say your site remains at the top of the engines for a few months perhaps even years each month you receive another 1000 visitors to your site, you have basically cut the cost of each surfer hitting your site down to less than a penny per hit (not taking into account bandwidth costs obviously).

    Now, on the other hand, you want to attract 1000 visitors from your chosen keywords via the PPC engines, most Pay Per Click search engines have a minimum bid amount of $0.05c per hit so right away in your first month, you could receive a potential 10k hits however, as most of you who have already tried your hand at the PPC engines will know, getting 10k hits for one or more keyword at a cost of $0.05 is hard to do, in fact, some would say almost impossible. Non the less let us keep going with this minimum bid amount for the time being.

    Immediately, you can see that you are already restricted to the actual amount of traffic you can receive from the PPC results to 10k hits however, this isn’t the case with the SEO traffic, you could potentially hit your top chosen keyword and stay there until another site out-optimizes you or, your site needs to be optimized again.

    Ultimately, the reasons you will choose over one or the other will be for either ease of traffic generation, PPC will allow you to gain almost instantaneous targeted traffic form the second you open your PPC account up until the point when your account funds empty whilst, SEO work will give you long term targeted traffic over time and, in most instances, this SEO traffic can last for years making the cost of the initial SEO work minimal.

    In Closing..

    Search Engine Optimization can last you years and years whilst Pay Per Click results can diminish in a relatively short amount of time depending on the amount of bid needed to achieve top listings.

    However lets look at a third option, using both PPC results and SEO results in conjunction with each other to minimize the traffic you lose from your SEO work and, to minimize the traffic you lose from your PPC results this will afford you the time to see what works with your Search Engine Optimized sites whilst being able to play with the targeting of keywords on your PPC traffic, once you have both types of search engine figured out, you can put them both together and use them to increase the traffic to your site for years to come.

    Article written by Lee

  • The Curse Of The Broken Image

    Date: 2011.02.24 | Category: WebDesign | Response: 0

    At one time or another we have all done it, uploaded a new site only to find out that when we view the page in our browser one of the images are broken because we either didn’t link it properly or the filename wasn’t named correctly on our server.

    We may think that this is a bad thing however, recent experimentation on my part has found quite the opposite.

    We all search for new ways of getting the surfer to visit our sponsors in order to have them buy a membership and earn us a nice little slice of $$$ in the process but, what methods could we use that quite simply are overlooked?

    One such method is that of utilizing broken images, lets be honest here, no-one wants broken image son their site, it makes the site look unprofessional but, lets think about this, on a page full of banners and graphics all nested together neatly on an FPA (Full Page Ad) what’s more likely to grab the surfers attention, a completely loaded heavy bandwidth graphic or a single solitary broken image placed in a such a spot the surfer actually notices it?

    Makes you think doesn’t it. Our recent testing of this method has actually shown that our CTR (Click Through Ratio) rose slightly on a couple of sponsors we used this method on all of the sites in question had nice looking text and banners with corresponding thumbnails leading tot he full size image but, upon deliberately leaving an image off the server thus resulting in the ‘appearance’ of an erroneous broken graphic the spot where that image should have been was were a great deal of our click throughs came from.

    I am not saying you should immediately go and delete all graphics from you pages but, a single, well placed broken image on a free site can work wonders in getting traffic off your site and to your sponsors pay site tour which ultimately, is what we want to happen at the EARLIEST possible time.

    Article written by Lee

  • The Lost Traffic Source – Picture Posts

    Date: 2011.02.24 | Category: Pic Posts | Response: 0

    Back in the day, pic posts were all the rage, taking less time for webmasters to build for and, less time maintaining however, in recent times we seldom hear of these traffic sources as more and more webmasters turn to the TGP’s in an effort to get traffic ‘quantity’ over ‘quality’ my personal feelings are that the pic posts of the past will soon start to make a comeback.

    The Basics.

    So, we know the history behind the Pic Post but what about the dynamics? How exactly do they work? That’s easy, you take an image, give it a HTML page and add the recip to the PP (Pic Post) you are submitting the page / image too along with a banner leading to your site. At this time, it would be prudent to add that, if you will be submitting to PP’s on a regular basis, you might also like to add the url to your main domain on the image in the lower right or left hand corner.

    One other thing to take into consideration when using PP’s is that softcore is often better. With the readily available hardcore content on TGP’s the chances are, that the surfer has decided to visit the PP because there is not as much widely available hardcore content.

    Cost Balance.

    In addition to the relatively minimal amount of work required to create pages for the PP’s there are several other factors that make them much better then TGP’s and Free Sites the main one is that bandwidth usage is relatively low for a start, not to mention the fact that you can knock out hundreds / thousands of PP pages a day compared to the tens / hundreds of TGP and Free Sites.

    Also, when looking at cost, the one factor people often forget is the submission time and, unlike the TGP model of traffic the vast majority of PP’s will actually encourage or, in some instances, prefer automated submissions, this means that you can target hundreds of PP’s in less time than it takes to submit a TGP or Free Site.

    Marketing Basics.

    Of course, as with any type of site that we build as webmasters, the main reason that surfers will visit our sponsors is the marketing ‘spin’ we give them, this is where the use of Alt Tags and Text can become extremely useful, bearing in mind, the surfer can already see the ‘larger picture’ on your PP page by utilizing Alt Tags and other SEO methods you can actually entice the surfer to click through to your sponsor and, get some very good rankings in the search engines at the same time.

    Housekeeping.

    Now you have started to use Picture Posts, as with any form of traffic you need to ensure you keep ‘accurate’ records of who is and who isn’t accepting your pages along with details stats on the amount of traffic each PP sends you and, you ultimately send off to your sponsors site.

    Adjust everything you can. trial and error plays a MAJOR part in using PP’s just because you use softcore images one day and get 1000 hits to the page you created, that doesn’t mean that by using hardcore images the next day you will receive more or, less traffic, also being niche specific, unlike with the TGP game can have its disadvantages, use images that appear at first glance, to be ‘generalized’ then, over time adjust your advertising methods to reflect the niche of surfer you are targeting.

    In addition to keeping records of the pages you create for the picture posts, you should also be sure to keep a record of where else you have used them, just because you are using them for the PP’s specifically, this doesn’t mean that you can not interlink these pages together and get some traffic of your own to them, does it?

    Hopefully this article has given you some more insight into the world of the Picture Post and, if you try an experiment with this little known source of quality traffic, I feel certain you will be pleasantly surprised with the results you see.

    Article written by Lee

  • Converting Chinese Adult Traffic – The Niche Paysites That Work

    Date: 2011.02.21 | Category: Traffic | Response: 0

    Converting Chinese Adult Traffic – The Niche Paysites That Work.

    To date we have looked at most of the types of country specific traffic available to webmasters every day and, more importantly, which niches they convert on. In this article we will finish up our look at the foreign traffic and how profitable it can be with the final main country on the net – China.

    Chinese Adult Surfers – Quick Analysis.

    As with Japanese traffic, most webmasters seem to think that Chinese speaking adult surfers are pretty worthless when it comes to buying porn memberships and this is true – If you do not know what niches to offer your Chinese surfers that said, we found some interesting results given the three (3) month test we had on our Chinese traffic and, the results we observed might just make you change your mind about sending the Chinese surfers you get to your sites off to traffic trades.

    Chinese Porn Surfers – The Niches.

    Because we knew relatively little about the Chinese traffic we had on our sites we decided that, rather send it to a specific language hub site we would send it to our main hub page and, allow the Chinese surfers to filter themselves through to the sites that interested them on their own, to our surprise, only 40% of the traffic we knew was Chinese chose to filter itself to a Chinese specific language hub site, the remainder, choosing to go through our main English hub site.

    Fetish.

    Across all of the niches we offered the surfers, the Fetish niche was the one that converted most consistently month after month and, more than converted, actually retained members once they had signed up to the paysites.

    Gay.

    Unlike the traffic we had that was Japanese a lot of the Chinese traffic that hit our site decided to visit the Gay niche paysites we had to offer them and, there was less than 10% difference between the Fetish and Gay signups we observed month after month. Quite a difference when compared to surfers from Japan and, a pleasant one at that.

    Amateur.

    Specifically teen amateur niche sites or, at least sites that had amateur looking teen modes on the tours also did well on our Chinese traffic however, the conversion ratios for the recurring programs didn’t do as well as what we had expected with most memberships being cancelled within 2 months. Non the less, the Chinese surfers did join and most of them re-billed in excess of 1 month before leaving the site.

    Teen.

    As mentioned above, it would appear that Chinese porn surfers are looking for teen niches again though, the teen paysites we used featured amateur looking teens prominently on the pages so whether this was a choice on the surfers part to join the teen sites or, because we only offered amateur looking teen paysites isn’t known.

    Asian.

    Unlike Japanese surfers, Chinese surfers also joined the Asian niches that were offered up to them and, surprisingly enough, we noticed more signups to English Asian sites than localized Chinese pay sites.

    Chinese Adult Surfers – The Results.

    Overall we were very impressed with how the Chinese traffic we had flowing through our sites both acted and, converted on the sites that we offered them to join, especially since the Japanese traffic we had didn’t seem to like the gay niche sites yet Chinese traffic worked wonderfully on exactly the same gay niche pay sites.

    Across the board I think a revisit of Chinese traffic is needed with more options on our part when it comes to the teen and amateur niches especially just to see if the results we experienced thus far were unique or, are truly indicative of how the Chinese surfer navigates and joins pay sites.

    Article written by Lee

  • Robots.txt – Control The Robots That Crawl Your Sites

    Date: 2011.02.24 | Category: Search Engine Optimization, WebDesign | Response: 0

    By writing a structured text file you can indicate to robots that certain parts of your server are off-limits to some or all robots. It is best explained with an example:

    # robots.txt file for general use on web servers.

    User-agent: webcrawler
    Disallow:

    User-agent: googlebot
    Disallow: /

    User-agent: *
    Disallow: /cgi-bin
    Disallow: /logs
    The first line, starting with ‘#’, specifies a comment.

    The first paragraph specifies that the robot called ‘webcrawler’ has nothing disallowed: it may go anywhere.

    The second paragraph indicates that the robot called ‘googlebot’ has all relative URLs starting with ‘/’ disallowed. Because all relative URL’s on a server start with ‘/’, this means the entire site is closed off.

    The third paragraph indicates that all other robots should not visit URLs starting with /cgi-bin or /log. Note the ‘*’ is a special token, meaning “any other User-agent”; you cannot use wildcard patterns or regular expressions in either User-agent or Disallow lines.

    Two common errors:

    Wildcards are not supported: instead of ‘Disallow: /tmp/*’ just say ‘Disallow: /tmp’.
    You shouldn’t put more than one path on a Disallow line (this may change in a future version of the spec)
    Ultimately, without the use of robots.txt files on your servers/domains, you are risking a variety of potential problems including, unauthorized access to your cgi directory, unauthorized viewing of your site stats, possible spamming of the search engines by accidental crawling of doorway pages.

    One distinct advantage however of having a robots.txt file on your server is that, quite simply, you will be able to tell when and where your site has been indexed or potentially indexed as, all robots will automatically call for the robots.txt file BEFORE any other page on your server so, as long as you keep an eye open for any calls of this file, you can see who is knocking at your site for indexing purposes.

    Below is a robots.txt example that you can copy and paste into a text document to use on your own server:

    <!–Start Copy Below This Line–>

    User-agent: *
    Disallow: /cgi-bin
    Disallow: /logs

    <!–End Copy Above This Line–>

    The above will allow all spiders to crawl all of your site except the subdirectory’s ‘cgi-bin’ and ‘logs’ which, may be altered to suit any subdirectory’s you do not wish the spiders to crawl on your server.

    Article written by Lee

Premium Sponsors















Categories

Site Links