If you have an existing Public website in your SharePoint 2013 online and it gives an authentication popup when your try to browse it as a public website your site might be in an “Offline mode”.
To make it available anonymously you need to make the Site Online. Navigation to your public site and click on the “Website Offline” link of the top right menu (as shown in the screen below)
Click Continue if you get the below screen.
Next click on the “Make Online” button in the new popup.