If you compare with EU regulation, Japan has a different regulation against smoking. Of course smoking is not allowed on trains, buses and airplanes. However, it is allowed to smoke inside the restaurants, cafes and diners(of course there are some places which smoking is strictly prohibited).
EU has a strict regulations for smoking inside the building but not a strict regulation against on the streets. On the other hand, Japan has a strict regulation against smoking on the streets, especially around the large stations, business districts and shopping districts. These areas are designated as non-smoking areas.
I assume that the EU are keen on people's health rather than the cleanliness of the streets. On the other hand, Japan is much keen on the cleanliness of the streets. Therefore if the area is designated as non-smoking area, you can not smoke there, no matter you are walking or standing. Even using your personal ashtray is not allowed you to smoke at non-smoking area.
Therefore the authorities must create smoking areas for the smokers. This is also different from EU. EU just ban smoking at the bars but do not create any smoking area or provide ashtrays outside the bars. Then, no one carries any personal ashtray, therefore the ashes and the cigarettes have to be thrown on the streets or the bars have to set up their ashtrays outside.
If you are smoker and going to Japan, I recommend to buy the portable ashtray at 100 yen shop and also watch out smoking on the streets.