Backport #22566
Change the mailer interface to prevent the leaking of possible hidden
email addresses when sending to multiple recipients.
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Backport #21945
Unfortunately the fallback configuration code for [mailer] that were
added in #18982 are incorrect. When you read a value from an ini section
that key is added. This leads to a failure of the fallback mechanism.
Further there is also a spelling mistake in the startTLS configuration.
This PR restructures the mailer code to first map the deprecated
settings on to the new ones - and then use ini.MapTo to map those on to
the struct with additional validation as necessary.
Ref #21744
Signed-off-by: Andrew Thornton <art27@cantab.net>
The recovery, API, Web and package frameworks all create their own HTML
Renderers. This increases the memory requirements of Gitea
unnecessarily with duplicate templates being kept in memory.
Further the reloading framework in dev mode for these involves locking
and recompiling all of the templates on each load. This will potentially
hide concurrency issues and it is inefficient.
This PR stores the templates renderer in the context and stores this
context in the NormalRoutes, it then creates a fsnotify.Watcher
framework to watch files.
The watching framework is then extended to the mailer templates which
were previously not being reloaded in dev.
Then the locales are simplified to a similar structure.
Fix#20210Fix#20211Fix#20217
Signed-off-by: Andrew Thornton <art27@cantab.net>
This bug affects tests which are sending emails (#20307). Some tests reinitialise the web routes (like `TestNodeinfo`) which messed up the mail templates. There is no reason why the templates should be loaded in the routes method.
* `PROTOCOL`: can be smtp, smtps, smtp+startls, smtp+unix, sendmail, dummy
* `SMTP_ADDR`: domain for SMTP, or path to unix socket
* `SMTP_PORT`: port for SMTP; defaults to 25 for `smtp`, 465 for `smtps`, and 587 for `smtp+startls`
* `ENABLE_HELO`, `HELO_HOSTNAME`: reverse `DISABLE_HELO` to `ENABLE_HELO`; default to false + system hostname
* `FORCE_TRUST_SERVER_CERT`: replace the unclear `SKIP_VERIFY`
* `CLIENT_CERT_FILE`, `CLIENT_KEY_FILE`, `USE_CLIENT_CERT`: clarify client certificates here
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
When Gitea is running as PID 1 git will occassionally orphan child processes leading
to (defunct) processes. This PR simply sets Setpgid to true on these child processes
meaning that these defunct processes will also be correctly reaped.
Fix#19077
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Start adding mechanism to return unhandled data
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Create pushback interface
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add Pausable interface to WorkerPool and Manager
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Implement Pausable and PushBack for the bytefifos
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Implement Pausable and Pushback for ChannelQueues and ChannelUniqueQueues
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Wire in UI for pausing
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add testcases and fix a few issues
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix build
Signed-off-by: Andrew Thornton <art27@cantab.net>
* prevent "race" in the test
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix jsoniter mismerge
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix conflicts
Signed-off-by: Andrew Thornton <art27@cantab.net>
* fix format
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add warnings for no worker configurations and prevent data-loss with redis/levelqueue
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Use StopTimer
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
It appears that several versions of sendmail require that the mail is sent to them with
LF line endings instead of CRLF endings - which of course they will then convert back
to CRLF line endings to comply with the SMTP standard.
This PR adds another setting SENDMAIL_CONVERT_CRLF which will pass the message writer
through a filter. This will filter out and convert CRLFs to LFs before writing them
out to sendmail.
Fix#18024
Signed-off-by: Andrew Thornton <art27@cantab.net>
* allways set a message-id on mails
* Add unit tests for mailer & Message-ID
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This PR registers requests with the process manager and manages hierarchy within the processes.
Git repos are then associated with a context, (usually the request's context) - with sub commands using this context as their base context.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Add settings to allow different SMTP envelope from address
Sometimes it may be advisable to hide or alias the from address on an SMTP mail
envelope. This PR adds two new options to the mailer to allow setting of an overriding
from address.
Fix#17477
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Unexport SendUserMail
* Instead of "[]*models.User" or "[]string" lists infent "[]*MailRecipient" for mailer
* adopt
* code format
* TODOs for "i18n"
* clean
* no fallback for lang -> just use english
* lint
* exec testComposeIssueCommentMessage per lang and use only emails
* rm MailRecipient
* Dont reload from users from db if you alredy have in ram
* nits
* minimize diff
Signed-off-by: 6543 <6543@obermui.de>
* localize subjects
* linter ...
* Tr extend
* start tmpl edit ...
* Apply suggestions from code review
* use translation.Locale
* improve mailIssueCommentBatch
Signed-off-by: Andrew Thornton <art27@cantab.net>
* add i18n to datas
Signed-off-by: Andrew Thornton <art27@cantab.net>
* a comment
Co-authored-by: Andrew Thornton <art27@cantab.net>
* Make sure that sendmail processes register with the process manager
* Provide a timeout for these (initially of 5 minutes)
* Add configurable value and tie in to documentation
* Tie in to the admin config page.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Mail issue subscribers, rework the function
* Simplify a little more
* Fix unused variable
* Refactor mail delivery to avoid heavy load on server
* Avoid splitting into too many goroutines
* Fix comments and optimize GetMaileableUsersByIDs()
* Fix return on errors
* move all mail related codes from models to modules/mailer
* fix lint
* use DBContext instead Engine
* use WithContext not WithEngine
* Use DBContext instead of Engine
* don't use defer when sess.Close()
* move DBContext to context.go and add some methods
* move mailer from modules/ to services
* fix lint
* fix tests
* fix fmt
* add gitea copyright
* fix tests
* don't expose db functions
* make code clear
* add DefaultDBContext
* fix build
* fix bug
* Add SUBJECT_PREFIX mailer config option
* Add space between subject prefix and subject (Change from Gogs)
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Panic don't fatal on create new logger
Fixes#5854
Signed-off-by: Andrew Thornton <art27@cantab.net>
* partial broken
* Update the logging infrastrcture
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Reset the skip levels for Fatal and Error
Signed-off-by: Andrew Thornton <art27@cantab.net>
* broken ncsa
* More log.Error fixes
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Remove nal
* set log-levels to lowercase
* Make console_test test all levels
* switch to lowercased levels
* OK now working
* Fix vetting issues
* Fix lint
* Fix tests
* change default logging to match current gitea
* Improve log testing
Signed-off-by: Andrew Thornton <art27@cantab.net>
* reset error skip levels to 0
* Update documentation and access logger configuration
* Redirect the router log back to gitea if redirect macaron log but also allow setting the log level - i.e. TRACE
* Fix broken level caching
* Refactor the router log
* Add Router logger
* Add colorizing options
* Adjust router colors
* Only create logger if they will be used
* update app.ini.sample
* rename Attribute ColorAttribute
* Change from white to green for function
* Set fatal/error levels
* Restore initial trace logger
* Fix Trace arguments in modules/auth/auth.go
* Properly handle XORMLogger
* Improve admin/config page
* fix fmt
* Add auto-compression of old logs
* Update error log levels
* Remove the unnecessary skip argument from Error, Fatal and Critical
* Add stacktrace support
* Fix tests
* Remove x/sync from vendors?
* Add stderr option to console logger
* Use filepath.ToSlash to protect against Windows in tests
* Remove prefixed underscores from names in colors.go
* Remove not implemented database logger
This was removed from Gogs on 4 Mar 2016 but left in the configuration
since then.
* Ensure that log paths are relative to ROOT_PATH
* use path.Join
* rename jsonConfig to logConfig
* Rename "config" to "jsonConfig" to make it clearer
* Requested changes
* Requested changes: XormLogger
* Try to color the windows terminal
If successful default to colorizing the console logs
* fixup
* Colorize initially too
* update vendor
* Colorize logs on default and remove if this is not a colorizing logger
* Fix documentation
* fix test
* Use go-isatty to detect if on windows we are on msys or cygwin
* Fix spelling mistake
* Add missing vendors
* More changes
* Rationalise the ANSI writer protection
* Adjust colors on advice from @0x5c
* Make Flags a comma separated list
* Move to use the windows constant for ENABLE_VIRTUAL_TERMINAL_PROCESSING
* Ensure matching is done on the non-colored message - to simpify EXPRESSION
* Create log mailer for testing email settings
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Switch on the log mailer for the integration tests
This ensures that the sending mail process works
Signed-off-by: Andrew Thornton <art27@cantab.net>
* rename the from user for mysql/mssql
* rename log sender to dummy sender
* update the integration tests
* Add support for extra sendmail arguments
* Sendmail args to exec.command should be a list
* Add go-shellquote package
* Use go-shellquote lib for parsing Sendmail args
* Only parse if sendmail is configured
* Fix sending mail with a non-latin display name. #2102
Signed-off-by: Rémi Saurel <contact@remi-saurel.com>
* Take into account the possibility that setting.MailService.From is in `name <email@address>` format. #2102
Signed-off-by: Rémi Saurel <contact@remi-saurel.com>
* Send mails as HTML as default. Setting for send as plain text.
* Add new option SendAsPlainText. remove EnableHTMLAlternative
* Send HTML mails as default
* Add html check if html2text should be performed
* Send only multipart or plain. Add deprication warning for ENABLE_HTML_ALTERNATIVE
* Still use ENABLE_HTML_ALTERNATIVE for backward compatibility
* Changed to not ignore html2text errors
This change corrects a few logging issues:
* Standardized formatting errors with '%v'.
* Standardized failure warning word usage.
* Corrected an instance of using the standard log library when
the gitea log library should be used instead.
* Implemented sendmail. This piggybacks on existing configuration to keep the change simple
* Changed privicy of new sendSMTP and sendSendmail functions
* Fixed Lint errors
* Seperated SMTP and sendmail into their own senders
* Making new structs private as they should not be used externally now
* Added sendmail setting to ini file
* Minor code cleanup
* Do not write HTML in text/plain mail part
Fixes#2928
* Pass text/plain first, text/html second
* Do not send plain/text email if html2text failed (untested)
Yarden Shoham
zeripath
delvh
KN4CK3R
Clar Fon
6543
Garionion
luzpaz
guillep2k
Lunny Xiao
kolaente
mrsdizzie
Lanre Adelowo
Jonas Bröms
Rémi Saurel
Bo-Yi Wu
Ethan Koenig
Gabriel Jackson
Philip Couling
Sandro Santilli
Rémy Boulanouar
Unknwon
Hongcai Deng