Commit Graph

52 Commits

Author SHA1 Message Date
Andrew Dolgov b0fc248c05
add tracing to UrlHelper 2023-04-09 23:26:51 +03:00
wn_ 0dbed700ef Merge remote-tracking branch 'origin/master' into feature/php-7.4-stuff 2022-08-15 10:43:14 +00:00
Andrew Dolgov 7d77edd1fb
amend logic flow to fix phpstan warning in previous 2022-08-15 07:59:24 +03:00
wn_ 830a20debf Handle 'file_get_contents()' failure in 'UrlHelper::fetch()'. 2022-08-14 16:52:44 +00:00
wn_ 57e31fe5a7 Handle valid HTTP responses with no response body. 2022-08-14 16:39:03 +00:00
wn_ 7567676ed8 Remove a PHP < 7.1 branch in UrlHelper. 2022-08-12 14:16:40 +00:00
wn_ 93fd85df6f Switch to direct type declarations of class properties. 2022-08-12 14:13:26 +00:00
Andrew Dolgov 1b3e655f89 use CURLAUTH_BASIC by default for password-protected feeds, keeping
CURLAUTH_ANY as a fallback in case we got a 403.
2022-05-23 08:43:04 +03:00
Siemenskun 3406a16025 Fix typo 2022-05-22 02:02:56 +03:00
Siemenskun d33d026b12 Fix xml parsing error
Move re-requesting logic before parsing response body, otherwise it puts HTTP headers into XML body
2022-05-22 01:46:46 +03:00
Andrew Dolgov 385da287d8 rewrite_relative: deal with undefined path warning 2022-03-22 19:43:32 +03:00
Andrew Dolgov 0345e9d3f6 rewrite_relative: use isset() to check for relative path 2022-03-22 16:18:22 +03:00
Andrew Dolgov e35a4a1306 tests: add stub autoloader, add a few more rewrite_relative tests 2022-03-22 14:32:32 +03:00
Andrew Dolgov 1c4f7ab3b8 * add phpunit as a dev dependency
* add some basic tests for UrlHelper::rewrite_relative()
 * fix UrlHelper::rewrite_relative() to work better on non-absolute
   relative URL paths
2022-03-22 12:24:31 +03:00
Andrew Dolgov 168dc6fe57 rewrite_relative: prevent php warning when checking for unset content type in EXTRA_SCHEMES_BY_CONTENT_TYPE 2022-02-18 16:44:03 +03:00
Andrew Dolgov 74a247fc5c rewrite_relative: whitelist specific schemes for URLs with 'known' content-types i.e. specified for enclosures 2022-02-17 22:38:38 +03:00
Andrew Dolgov 1818fc11a5 fetch: return HTTP code when no curl_error() is available 2021-12-23 17:32:44 +03:00
Andrew Dolgov 204f92b926 urlhelper: add debugging output for download attempts 2021-11-24 08:19:04 +03:00
Andrew Dolgov 3323ae78ce * sql_bool_to_bool: make parameter nullable
* errorhandler: don't try to truncate null strings
 * UrlHelper::rewrite_relative: fix undefined offset warnings for URLs
that lack schema/host (data: etc)
2021-11-20 16:11:44 +03:00
wn_ fb208bb136 Fix a PHPStan warning in 'UrlHelper::rewrite_relative()'. 2021-11-15 03:28:17 +00:00
Andrew Dolgov 931a7533ce adjust some return types in urlhelper 2021-11-14 20:53:30 +03:00
wn_ d3a81f598b Switch class properties from PHP typing to PHPDoc for compatibility with PHP < 7.4.0 2021-11-12 21:17:31 +00:00
wn_ 2d5603b196 Address PHPStan warnings in 'classes/diskcache.php'. 2021-11-11 22:07:32 +00:00
wn_ 7a919a79d7 Fix some additional PHPStan warnings in UrlHelper. 2021-11-11 11:12:40 +00:00
wn_ 4cc3374f9f Initial go at PHPStan rule level 6. 2021-11-10 21:38:25 +00:00
wn_ 9dac9c5a0d Address PHPStan warnings in 'classes/urlhelper.php'.
Intentionally skipping the line 66 one for now; adding an 'is_array' check clears the warning, but there's a larger topic of how to handle an invalid '' that doesn't result in an array.

------ ---------------------------------------------------------------------
Line   classes/urlhelper.php
------ ---------------------------------------------------------------------
66     Offset 'path' on array{scheme: string} in isset() does not exist.
165    Parameter #2 $associative of function get_headers expects bool, int
        given.
167    Parameter #2 $associative of function get_headers expects bool, int
        given.
278    Negated boolean expression is always true.
309    Negated boolean expression is always true.
------ ---------------------------------------------------------------------
2021-11-01 21:10:27 +00:00
Andrew Dolgov 2c931df77c remove SELF_USER_AGENT custom constant, replaced with configurable Config::HTTP_USER_AGENT / Config::get_user_agent() 2021-08-23 10:56:31 +03:00
Philip Klempin 1d4d3bc49c Fix operator precedence 2021-06-18 12:52:29 +02:00
Andrew Dolgov 800ebd6373 revise previous a little bit more 2021-06-18 11:52:06 +03:00
Andrew Dolgov 69f261c41d revise previous a little bit 2021-06-18 11:30:11 +03:00
Andrew Dolgov e9c062a189 UrlHelper::rewrite_relative():
- support invoking specifying owner URL element/attribute
 - restrict mailto/magnet/tel schemes for A href
 - allow some data: base64 image types for IMG src

Sanitizer::sanitize():

 - when checking href and src attributes, pass element tagname and attribute to rewrite_relative()
2021-06-18 11:20:57 +03:00
Philip Klempin b3bedd0a94 Skip URI base on ALLOWED_RELATIVE_SCHEMES in rewrite_relative 2021-06-16 15:24:15 +02:00
Andrew Dolgov 326850845d UrlHelper::rewrite_relative: don't try to feed NULL to with_trailing_slash() 2021-05-21 17:10:32 +03:00
Andrew Dolgov dff479af64 feeditem_atom: support xml:base for enclosures and entry content
UrlHelper::rewrite_relative: use base URL path if relative url path is not absolute (experimental)
2021-05-21 15:39:41 +03:00
Andrew Dolgov 86300a0ca8 add urlhelper to extract youtube video id from url 2021-05-07 07:37:27 +03:00
Andrew Dolgov 4ddcd54e8d * limit progressfunction debugging to size quota exceeded notifications
* af_redditimgur: reparent generated iframes outside of post table
2021-03-13 11:18:59 +03:00
ltGuillaume 825e362f0e Fix "array offset on value of type null" for $error and $old_error
I tried applying to only $error and only $old_error, but both appear to be needed.

Log entries:
E_NOTICE (8) 	classes/urlhelper.php:464 	Trying to access array offset on value of type null
1. classes/urlhelper.php(464): ttrss_error_handler(8, Trying to access array offset on value of type null, classes/urlhelper.php, 464, [)
2. classes/rssutils.php(464): fetch([{"url":"https://some.url.rss","login":"","pass":"","timeout":15,"last_modified":"Sat, 31 Aug 2019 15:22:31 GMT"})
3. update.php(235): update_rss_feed(732, 1)
2021-03-06 20:33:23 +01:00
Andrew Dolgov 320503dd39 move version-related stuff to Config; fix conditional feed requests 2021-03-01 13:43:37 +03:00
Andrew Dolgov afc7142250 move all $fetch globals to UrlHelper 2021-02-28 10:12:57 +03:00
Andrew Dolgov 2ae0b7059f cleanup some defined-stuff 2021-02-23 09:01:27 +03:00
Andrew Dolgov 211f699aa0 migrate the rest into Config:: 2021-02-22 22:35:27 +03:00
Andrew Dolgov b6e1a5c91a fix several warnings reported by phpstan 2021-02-06 17:19:07 +03:00
Andrew Dolgov 403dca154c initial WIP for php8; bump php version requirement to 7.0 2021-02-05 23:41:32 +03:00
wn 95d0cb4953 Handle potential absence of a URL path in UrlHelper. 2020-12-12 10:28:53 -06:00
Andrew Dolgov 328d7b55c8 URLHelper: fix E_DEPRECATED error related to idn_to_ascii() 2020-11-14 15:13:35 +03:00
Andrew Dolgov c70e26db31 validate url: feed urlencoded() URL to filter_var() only 2020-09-28 19:46:31 +03:00
Andrew Dolgov 8fb2baecdc another hack for validation of URLs with invalid characters 2020-09-22 19:56:26 +03:00
Andrew Dolgov a897c4165b validate URLs: convert IDN to punycode before passing URL to filter_var() 2020-09-22 15:32:22 +03:00
Andrew Dolgov 6811d0bde2 use self:: in some places to invoke static methods from the same class 2020-09-22 14:54:15 +03:00
Andrew Dolgov b5710baf34 - don't fail on non-ascii characters when validating URLs
- fix IDN hostnames not being converted properly
2020-09-22 14:37:45 +03:00