There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this.
This is an extensive PR attempting to fix these issues.
1. The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping.
2. Whilst doing this we will prefer to use functions that create URLs over recreating them by hand.
3. All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future.
4. The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.)
5. The title generation in feeds is now properly escaped.
6. EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andrew Thornton <art27@cantab.net>
* DBContext is just a Context
This PR removes some of the specialness from the DBContext and makes it context
This allows us to simplify the GetEngine code to wrap around any context in future
and means that we can change our loadRepo(e Engine) functions to simply take contexts.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix unit tests
Signed-off-by: Andrew Thornton <art27@cantab.net>
* another place that needs to set the initial context
Signed-off-by: Andrew Thornton <art27@cantab.net>
* avoid race
Signed-off-by: Andrew Thornton <art27@cantab.net>
* change attachment error
Signed-off-by: Andrew Thornton <art27@cantab.net>
When create a new issue or comment and paste/upload an attachment/image, it will not assign an issue id before submit. So if user give up the creating, the attachments will lost key feature and become dirty content. We don't know if we need to delete the attachment even if the repository deleted.
This PR add a repo_id in attachment table so that even if a new upload attachment with no issue_id or release_id but should have repo_id. When deleting a repository, they could also be deleted.
Co-authored-by: 6543 <6543@obermui.de>
* chore: rewrite format.
* chore: update format
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
* chore: update format
Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
* chore: Adjacent parameters with the same type should be grouped together
* chore: update format.
* Add a storage layer for attachments
* Fix some bug
* fix test
* Fix copyright head and lint
* Fix bug
* Add setting for minio and flags for migrate-storage
* Add documents
* fix lint
* Add test for minio store type on attachments
* fix test
* fix test
* Apply suggestions from code review
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Add warning when storage migrated successfully
* Fix drone
* fix test
* rebase
* Fix test
* display the error on console
* Move minio test to amd64 since minio docker don't support arm64
* refactor the codes
* add trace
* Fix test
* remove log on xorm
* Fi download bug
* Add a storage layer for attachments
* Add setting for minio and flags for migrate-storage
* fix lint
* Add test for minio store type on attachments
* Apply suggestions from code review
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Fix drone
* fix test
* Fix test
* display the error on console
* Move minio test to amd64 since minio docker don't support arm64
* refactor the codes
* add trace
* Fix test
* Add URL function to serve attachments directly from S3/Minio
* Add ability to enable/disable redirection in attachment configuration
* Fix typo
* Add a storage layer for attachments
* Add setting for minio and flags for migrate-storage
* fix lint
* Add test for minio store type on attachments
* Apply suggestions from code review
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
* Fix drone
* fix test
* Fix test
* display the error on console
* Move minio test to amd64 since minio docker don't support arm64
* don't change unrelated files
* Fix lint
* Fix build
* update go.mod and go.sum
* Use github.com/minio/minio-go/v6
* Remove unused function
* Upgrade minio to v7 and some other improvements
* fix lint
* Fix go mod
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: Tyler <tystuyfzand@gmail.com>
Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util.
Fix#12339
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
* Use ID or Where to instead directly use Get when load object from database
* Apply suggestions from code review
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
* test: add current attachement responses
* refactor: check if attachement is linked and accessible by user
* chore: clean TODO
* fix: typo attachement -> attachment
* revert un-needed go.sum change
* refactor: move models logic to models
* fix TestCreateIssueAttachment which was wrongly successful
* fix unit tests with unittype added
* fix unit tests with changes
* use a valid uuid format for pgsql int. test
* test: add unit test TestLinkedRepository
* refactor: allow uploader to access unlinked attachement
* add missing blank line
* refactor: move to a separate function repo.GetAttachment
* typo
* test: remove err test return
* refactor: use repo perm for access checking generally + 404 for all reject
* display ui time with customize time location
* fix lint
* rename UILocation to DefaultUILocation
* move time related functions to modules/timeutil
* fix tests
* fix tests
* fix build
* fix swagger
* Add size column to attachment
Migrate attachments by calculating file sizes
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Calculate attachment size on creation
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Log error instead of returning error
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Improve release page ui by adding size
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Add documentation to a.MustSize()
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Add Attachment API
* repos/:owner/:repo/releases (add attachments)
* repos/:owner/:repo/releases/:id (add attachments)
* repos/:owner/:repo/releases/:id/attachments
* repos/:owner/:repo/releases/:id/attachments/:attachment_id
Signed-off-by: Jonas Franz <info@jonasfranz.de>
* Add unit tests for new attachment functions
Fix comments
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* fix lint
* Update vendor.json
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* remove version of sdk
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Fix unit tests
Add missing license header
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Add CreateReleaseAttachment
Add EditReleaseAttachment
Add DeleteReleaseAttachment
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Add filename query parameter for choosing another name for an attachment
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Fix order of imports
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Restricting updatable attachment columns
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* gofmt
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Update go-sdk
Replace Attachments with Assets
Signed-off-by: Jonas Franz <info@jonasfranz.de>
* Update go-sdk
Signed-off-by: Jonas Franz <info@jonasfranz.de>
* Updating go-sdk and regenerating swagger
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Add missing file of go-sdk
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Change origin of code.gitea.io/sdk to code.gitea.io/sdk
Update code.gitea.io/sdk
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Update swagger
Signed-off-by: Jonas Franz <info@jonasfranz.software>
* Update updateAttachment
* refactor struct's time to remove unnecessary memory usage
* use AsTimePtr simple code
* fix tests
* fix time compare
* fix template on gpg
* use AddDuration instead of Add
* use created & updated instead BeforeInsert & BeforeUpdate
* fix vendor checksum
* only show generated SQL when development mode
* remove extra update column updated_unix
* remove trace config
6543
Lunny Xiao
zeripath
Bo-Yi Wu
Antoine GIRARD
gdeverlant
Jonas Franz
Lauris BH
Ethan Koenig
Thomas Boerger
Philip Couling