[HINDI] What is robots.txt? | How Website Indexing Works | Information Gathering Tool?

Hello everyone. In this video I will be talking about a very interesting topic and concept, which is how web crawlers of search engines analyze a website content and index them in appropriate search results. We will see what does this "robots.txt" file actually does, its working, advantages and disadvantages, what does it contain, and how penetration testers can get help from this for their work in information gathering or vulnerability scanning as well. SO CLEAR THIS COMMON DOUBT AND GET STARTED!! --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Download my OFFICIAL APP to learn ethical Hacking FREE: https://play.google.com/store/apps/details?id=com.bittentech.learnapp Follow this link to join my WhatsApp group: https://chat.whatsapp.com/3brcdlXRDiIBApiTACLqyo For cool Techie merchandise visit: https://bitten.tech/shop Increase your GK with my Own Quiz App : https://play.google.com/store/apps/details?id=com.bittentech.quizz Follow me on Instagram : https://instagram.com/ansh.98 Like me on Facebook : https://linksad.net/qUUBB ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If you liked my video, please don't forget to press the like button and subscribe to my youtube channel. I will be posting videos on cyber security, ethical hacking and technology. Also some interesting tricks and techniques very soon so stay tuned and please SUBSCRIBE to my channel and thanks for watching....:-)