bjoernager/forgejo
bjoernager/forgejo
1
Fork 0
Code Issues Pull requests Activity

feat: include a default robots.txt to reduce the impact of crawlers (#7387)

Browse source 
- Add a strong strict default robots.txt, if one is not provided by the instance administrators.
- Remove code for the legacy public asset path, the error has been logged for a few releases already (existed since v1.21).
- Resolves forgejo/forgejo#923

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7387
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Reviewed-by: 0ko <0ko@noreply.codeberg.org>
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-committed-by: Gusted <postmaster@gusted.xyz>
This commit is contained in:
Gusted 2025-03-30 11:28:19 +00:00 • committed by Gusted
parent 51caba694a
commit bb4e1f426f
2 changed files with 88 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

3
cmd/web.go
View file

@ -198,9 +198,6 @@ func serveInstalled(ctx *cli.Context) error {
for fn := range publicFilesSet.Seq() {
log.Error("Found legacy public asset %q in CustomPath. Please move it to %s/public/assets/%s", fn, setting.CustomPath, fn)
}
if _, err := os.Stat(filepath.Join(setting.CustomPath, "robots.txt")); err == nil {
log.Error(`Found legacy public asset "robots.txt" in CustomPath. Please move it to %s/public/robots.txt`, setting.CustomPath)
}
routers.InitWebInstalled(graceful.GetManager().HammerContext())