Sign Up
Log In
Sign Up
Username:
*
Email:
*
Password:
*
Password confirmation:
*
or
Log In
Log In
or
Sign Up
Places
All Projects
Status Monitor
Collapse sidebar
Please login to access the resource
Web
pihole-ftl
pihole-ftl.changes
Overview
Repositories
Revisions
Requests
Users
Attributes
Meta
File pihole-ftl.changes of Package pihole-ftl
------------------------------------------------------------------- Wed Feb 21 04:51:35 UTC 2024 - pihole-suse-packages@smar.fi - Update to version 5.25.1: * Update embedded dnsmasq version to 2.90+1 * Fix spurious "resource limit exceeded" messages. * Update dnsmasq version to 2.90 * Update expected dnsmasq warnings * Reverse suppression of ANY query answer logging. * Add --dnssec-limits option. ------------------------------------------------------------------- Mon Feb 19 05:51:14 UTC 2024 - pihole-suse-packages@smar.fi - Update to version 5.25: * Update dnsmasq version to 2.90 * Update expected dnsmasq warnings * Reverse suppression of ANY query answer logging. * Add --dnssec-limits option. * Better allocation code for DS digest cache. * Better stats and logging from DNSSEC resource limiting. * Overhaul data checking in NSEC code. * Rework validate-by-DS to avoid DoS vuln without arbitrary limits. * Update EDE code -> text conversion. * Parameterise work limits for DNSSEC validation. * Fix error introduced in 635bc51cac3d5d7dd49ce9e27149cf7e402b7e79 * Measure cryptographic work done by DNSSEC. * Update NSEC3 iterations handling to conform with RFC 9276. * Update header with new EDE values. * Protection against pathalogical DNSSEC domains. * Update embedded dnsmasq version to 2.90test4 * Make --filter-rr=ANY filter the answer to ANY queries. * Tweak logging and special handling of T_ANY in rr-filter code. * Force-update embedded dnsmasq version. We are loosing the individual dnsmasq history of the ~ last year, however, given the multitude of merge conflicts and the fact that this code will soon(ish) be replaced by development-v6 (where the history is 100% intact), this isn't much of an issue * Update changed indentation of known DNSMASQ warning * =/== typo in last commit. * Behave better when attempting to contact unresponsive TCP servers. * Necessary changed to handle the most recent dnsmasq changes in FTL * Log truncated DNS replies. * Apply suggestions from code review * Add special non-interactive mode for the embedded sqlite3 engine accessible via "-ni" * Bump actions/stale from 8.0.0 to 9.0.0 * Change priorities such that special domains (Firefox and Apple at this time) can be explicitly allowed for some clients (per group assignments) while they stay blocked for all others in the network * Fix possible race-collision leading to a theoretical out-of-bounds read * Bump actions/checkout from 3.5.3 to 3.6.0 * Group dependabot PRs * Bump actions/upload-artifact from 3.1.2 to 3.1.3 * Bump actions/checkout from 3.4.0 to 3.5.3 ------------------------------------------------------------------- Wed Nov 22 05:19:19 UTC 2023 - pihole-suse-packages@smar.fi - Update to version 5.23: * Tweak conditional, add in missing `env:` * Update stale.yml * Read this, added requestor(s) to the ignore list https://oneminuteenglish.org/en/requestor-or-requester/ * Update dependabot.yml * Use env variable * Do not try to remove stale lables on PRs ------------------------------------------------------------------- Thu Jun 8 04:35:31 UTC 2023 - Samu Voutilainen <smar@smar.fi> - Include pi-hole during build to allow it to handle permissions of /etc/pihole directory. ------------------------------------------------------------------- Tue May 30 04:32:59 UTC 2023 - Samu Voutilainen <smar@smar.fi> - Refreshed patches - Removed patch fix-build.patch that was applied to upstream ------------------------------------------------------------------- Tue May 30 04:29:40 UTC 2023 - pihole-suse-packages@smar.fi - Update to version 5.23: * Remove traces of ABP_CSS * Remove code duplication found in gravit.sh gravity_ParseFileIntoDomains() * Update src/tools/gravity-parseList.c * Rename src/{gravity-tools.* => tools/gravity-parseList.*} * Update adlist.date_updated in parseList command * Adding anchors to false_positives_regex * Improving the comments * Do not consider false positives as invalid domains * Allowing underscore and hyfen in any position for gravity parseList * Do not run ARP scans in networks where the kernel knows that ARP is not supported (e.g. Wireguard) * Do not try to scan for DHCP servers in network where the kernel knows that there is no broadcasting support (e.g. Wireguard) * Skip interfaces that are either down or are of loopback type * Ensure we are in lock-mode when printing the final result * Improve message when packet is rejected by wireguard interfaces * Improve deplay.sh script to check against exact matches instead of regex-matching the searched string against the entire collapsed array to avoid incorrect partial matches * Query IPv4-capable interfaces instead of packet-interfaces when scanning for DHCP servers * Modify logging in such a way that concurrent printing by the involved is prevented and add better error reporting when sending to interfaces is not working due to an error * Add capabilities check for feature dhcp-discover in the same way we already have it for arp-scan * Align % in reply rate column * Apply Pi-hole specific patches * Update SQLite3 to 3.42.0 * Further reduce memory requirements by factor 10x (if not in -x mode) * Reduce memory requirements by factor 4x * Exit early if insufficient memory is available, perform as many interface scans as possible under these conditions * Give reply rate in percent instead of showing the reply matrix * Add arp-scan -xtreme mode for very unreliable connections * Optimize thread_data structure and store a thread-local copy of the interface name * Add capabilities check for CAP_NET_RAW (root always has it) * Log more verbose human-readable error string if available * Clearly log when scanning interfaces failed * Interface names can be up to 16 bytes long. Docker bridge interfaces actually use this space so we need to reserve enough space here * Always skip the loopback interface, also in "-a" mode * Only print progress if it has changed. Otherwise, print "." as hearthbeat * Scale progress percentage according to number of addresses to be scanned by the individual threads * Consolidate output in main process * Print different warnings if we received multiple replies from (apparently) the same device or if we received replies for the same address from different MAC addresses * Print progress in verbose arp-scanning mode * Use OVER constant instead of carridge return * Spellcheck correction * Skip ABP extended CSS selectors (port of core PR #5247) * Re-apply Pi-hole specific Lua patches * Update embedded Lua to 5.4.6 * Include hostnames (if available) * Add our own address to the scan results so we can detect IP conflicts also here * Use dedicated counters per MAC for a more accurate per-device reply matrix * Unify warning * Add pihole-FTL arp-scan [{-v,-a}] * Move dhcp-discover into a dedicated "tools" target * Do not log running out of disk space when the disk occupation is > 100%. We are seeing this with docker deployments on macOS hosts. It is a band-aid fix, however, it also seem to be the only thing we can do given that docker didn't fix this in nearly two years now. * Also analyze UDP reply headers * Add extra debugging output * Add header analysis also in tcp_key_recurse to fix an issue with wrong upstream servers being attributed to DNSSEC-related queries when multiple upstream servers are defined (e.g. conditional forwarding) * Apply Pi-hole Lua patches * Update embedded Lua to 5.4.5 * Use env variable * Run seperate job to trigger removal on comments * Bump actions/checkout from 3.5.0 to 3.5.2 * Trigger stale workflow on issue comments to remove stale label immediately * Apply the same logic also for reverse lookups (PTR) * Explicitly set INSECURE status for replies received either from upstream (if they are not already validated as SECURE) or from cache. This is a direct consequence from the previous commit. * Initial DNSSEC status should be UNSPECIFIED * Analyse pseudeoheader before it might get stripped off * Log if EDNS header is NULL and we are in debug mode * Only try to interpret EDNS EDE when EDE data is available * Ignore possible EXTRA-TEXT field in EDNS0 EDE data * Use AD bit for IN/SECURE and EDE in SERVFAIL when prox for BOGUSy-dnsmasq option is used * Implement EDNS(0) EDE * Simplify EDNS handling code and also interpret replies received from upstream * Allow TLD blocking using ABP style (port of core PR #5240) * Add a few micro-optimizations to enhance speed of the parseList function and transform FQDN to domains. They are equivalent in this context but now they are not considered invalid any longer * Store in the database instead of into a temporary file * Enhance speed for ABP patterns (don't try to match domains when the line starts in "|") * Only match full lines in input file * Add gravity parseList funtion to FTL * Bump actions/checkout from 3.4.0 to 3.5.0 * Bump actions/stale from 7.0.0 to 8.0.0 * Correct declaration for blockingstatus variable. * Correct declaration for query_blocked(). ------------------------------------------------------------------- Fri Mar 24 05:24:42 UTC 2023 - Samu Voutilainen <smar@smar.fi> - Added patch fix-build.patch * Fixes build on Tumbleweed. ------------------------------------------------------------------- Fri Mar 24 04:32:12 UTC 2023 - pihole-suse-packages@smar.fi - Update to version 5.22: * Bump actions/checkout from 3.3.0 to 3.4.0 * Update dnsmasq version to pi-hole-v2.89-9461807 * Add RISC-V 64-bit support and builds * Add .codespellignore file to fix spell-checker action * Remove limitation on --dynamic-host. * Fix DHCPv6 "use multicast" response which previously failed to set the message type correctly. * Allow configuring filter-A/AAAA via dbus. * Generalise cached NXDOMAIN replies. * Set the default maximum DNS UDP packet size to 1232. * Fix possible SEGV when no servers defined. * Fix --rev-server option. It was broken in 1db9943c6879c160a5fbef885d5ceadd3668b74d when resolving upstream servers by name was extended to --rev-server without accounting for the fact that re-using one and the same upstream server for each of the x.y.z.in-addr.arpa is actually a wanted feature * Avoid undefined behaviour with the ctype(3) functions. * Put version.ftl also behind new no-ident config option * Apply Pi-hole SQLite3 patches * Update embedded SQLite3 engine to version 3.41.1 * Remove last traces of temporarily added benchmarking tools. Also remove the hint about ABP domains, this can easily be checked in gravity * Remove debugging timing output * Set abp_domains = 1 during the CI tests. * Use property "abp_domains" from info table to decide whether ABP blocking is to be used or not. Also log when FTL enabled ABP-style blocking * Add timing for ABP style detection * Update src/database/gravity-db.c * Fix spellcheck to get things deployed * Fix handling of rare (but possible) gravity database issues such as "list not available" * Do not use a new option but instead automatically detect if ABP-style domains are present in the database. This ensures that this addition comes at no extra costs to any installs using pure HOSTS-style adlists. * Add ABP format blocking support for gravity. Note that the option needs to be switched on by setting GRAVITY_ABP_STYLE=true in pihole-FTL.conf to avoid running this computationally expensive task on the vast majority of user databases only fed from properly formatted HOSTS lists. Gravity can enable the setting when it detects ABP format automatically. * Update dnsmasq version to 2.89 * Update dnsmasq version to 2.89rc1 * New syntax: querytype=A accepts now also a list (like querytype=A,AAAA,MX). You can use the exclamation mark as before for inversion (querytype=!A) matches everything BUT type A queries. This has now been extended to be able to invert a list, too (like (querytype=!A,AAAA matches everything BUT A and AAAA queries) * Add --no-ident option. * Print regex type hints only in debug mode * Allow selection of multiple query types in regex extension, like "abcabc;querytype=HTTPS,SVCB" * Fix bug which can break the invariants on the order of a hash chain. * Fix cosmetic big in dump_cache_entry() * Log all cache internal errors. * If we hit a cache internal error, log the entry we failed to remove. * Bump benoitchantre/setup-ssh-authentication-action from 1.0.0 to 1.0.1 * Remove gray color in help output * Bump actions/download-artifact from 3.0.1 to 3.0.2 * Bump actions/upload-artifact from 3.1.1 to 3.1.2 * Don't run the `Transfer Builds to Pi-hole Server...` step for dependabot PRs * Bump actions/checkout from 3.2.0 to 3.3.0 * Don't run the `Transfer Builds to Pi-hole Server...` step for dependabot PRs * More detailed comments * Add action to close stale PR * Always store time we start to save to the database (not only in debug mode). This avoids errorneous timing reports in case of errors. * Bump actions/stale from 6.0.1 to 7.0.0 * Be honest * Bump actions/checkout from 3.1.0 to 3.2.0 * Use github cli to sync master back to development * Add merge conflict workflow ------------------------------------------------------------------- Sat Jan 7 06:53:48 UTC 2023 - Samu Voutilainen <smar@smar.fi> - Made /var/log/pihole to be group writable, to allow php-fpm as nobody:pihole to write a log inside here. ------------------------------------------------------------------- Sat Jan 7 05:57:41 UTC 2023 - Samu Voutilainen <smar@smar.fi> - Add fortify_source_3.patch * Applied only for Tumbleweed. * Avoid error due redifinion of _FORTIFY_SOURCE - Ghost own /run/pihole ------------------------------------------------------------------- Fri Jan 6 06:01:25 UTC 2023 - Samu Voutilainen <smar@smar.fi> - Added link_against_readline.patch * Fixes build ------------------------------------------------------------------- Sat Dec 31 11:11:08 UTC 2022 - Samu Voutilainen <smar@smar.fi> - Use shared libraries instead of static ------------------------------------------------------------------- Sat Dec 31 09:30:37 UTC 2022 - pihole-suse-packages@smar.fi - Update to version 5.20: * Update embedded dnsmasq to v2.88 * Add posix-timezone and tzdb-timezone DHCPv6 options. * Review comments * Exit immediately after running dnsmasq-test * Fix logic for status code parsing * Fix incorrect DNSSEC-related warning during history import ------------------------------------------------------------------- Sat Oct 2 05:14:39 UTC 2021 - Samu Voutilainen <smar@smar.fi> - Only enable malloc error muting on Tumbleweed. ------------------------------------------------------------------- Sat Oct 2 05:00:56 UTC 2021 - Samu Voutilainen <smar@smar.fi> - Use -Wno-error=suggest-attribute=malloc as build flag to fix Tumbleweed building. -------------------------------------------------------------------- Sat Oct 2 04:49:29 UTC 2021 - pihole-suse-packages@smar.fi - Update to version v5.10.2 + Move SFTP xfer to happen before attach to release. Seeing some SSL errors in the github-action-publish-binaries action. + Fix REPLY_ADDR{4,6} address overwriting for pi.hole and <hostname> + Fix confusion in DNS retries and --strict-order. + Fix FTBFS when CONNTRACK and UBUS but not DNSSEC compile options selected. + dnsmasq_time: avoid signed integer overflow when HAVE_BROKEN_RTC + Do not fail hard when rev-server has a non-zero final address part + Update embedded dnsmasq version to 2.87test3 ------------------------------------------------------------------- Thu Sep 30 03:52:59 UTC 2021 - Samu Voutilainen <smar@smar.fi> - Removed unnecessary patches: + ftl-2.8.1-build-fix.patch + ignore-shmem.c-strncpy-error.patch -------------------------------------------------------------------- Thu Sep 30 03:41:11 UTC 2021 - pihole-suse-packages@smar.fi - Update to version v5.10.1 + Fix specific NOERR/NXDOMAIN confusion. + Reduce code duplication by merging FTL_cache() into FTL_reply() + Also process automatically generated queries, e.g. for DNSSEC validation + Add option to suppress automatically generated DNSSEC queries from being analyzed and shown (legacy behavior) + Fix bug in 6860cf932baeaf1c2f09c2a58e38be189ae394de + Fix bug introduced in 6860cf932baeaf1c2f09c2a58e38be189ae394de + Don't print flags multiple times in debug mode. + Log client requesting automatically generated DS/DNSKEY queries explicitly as "pi.hole" + Further work from a0a3b8ad3e91db5181023fceea6732eb6c6f0759 + Connection track mark based DNS query filtering. + Use correct packet-size limit in make_local_answer() + Include EDNS0 in connmark REFUSED replies. + Rename replyt ype 11 DNSKEY -> DNSSEC + Add src/dnsmasq/pattern.c to src/dnsmasq/CMakeList.txt + Update SQLite engine to 3.36.0 + Also cancel other threads when terminating + Ensure API threads can be canceled asynchronously + Add limit of maximum threads to warning + Add explicit limit logging also in the second place. + If DELAY_STARTUP is set, we can delay earlier to have this option being useful for misbehaving fake hwclocks as well. + Correct domain search algorithm. + Analyze which upstream server sent us the reply + Store real over-time counts of forwarded queries. So far, we counted only the first server a query was sent to. + Change upstream associated with a query if it is different than the first server we sent a query to + Log resolution of pi.hole and hostname as "internal" instead of the last blocking reason (e.g. "gravity blocked"). + Tests: Debug messages do now include the port a client sent the query from + Add more debugging output to short-circuited replies + Fix automatic IP hostname responding for blocking modes NXDOMAIN, NODATA and NODATA-IPv6 + Simplify logic in FTL_make_answer() + Fix error in try to make outer SHM lock consistent on dead of previous owner + Initial changes for extended DNS error codes. + Rationalise --server parsing and datastructure building. + Deprecate DEBUG_DNSMASQ_LINES (now included in DEBUG_FLAGS) + Initial implementation of RFC-8914 extended DNS errors. + Implement Extended DNS Errors (ERE, RFC 8914) in FTL + Don't re-use datastructures for --address and --local. + Rationalise domain parsing for --rev-server and --domain. + Fix problem with re-allocation of serverarray. + Include EDE in telnet API getAllQueries + Tidy up interface to dbus and ubus modules. + Compiler warnings. + Fix trivial breakage of DBUS done by 85bc7534dae7711f6c82742feaa7dacb41af3f36 + Fix compiler warning. + Tidy up name buffer use in report_addresses(). + Treat failure of ubus_add_object() in ubus_init() as retry-able. + Revert "Treat failure of ubus_add_object() in ubus_init() as retry-able." + Fix ipset support. + Reduce memory footprint of FTL by 11%. We don't store the rowid of a query in memory because we don't really need that. + Further reduce memory footprint of FTL by about 12%. We don't store the char pointer of the extended DNS errors because we can get this at any time. + Reuse workspace bit in struct server ->flags. + Allow wildcards in domain patterns. + Fix oversight in build_server_array(). + Rationalise SERV_MARK use. + Modify and propagate changed lease. + Hide "unknown" EDE in API + Implement special handling of the Mozilla canary domain to disable Firefox auto-DoH + Initialize over-time data only after a possible startup delay + Tidy domain parsing, make --server=/*/1.2.3.4 equivalent to --server=1.2.3.4 + Make --rebind-localhost-ok apply to :: and 0.0.0.0 + Support IPv6 in --bogus-nxdomian and --ignore-address + Fix order of calls to resize-packet() and add_pseudoheader(). + Add calls to dump internally generated answers for dumpmask=0x0002 + Fix logical error in d0ae3f5a4dc094e8fe2a3c607028c1c59f42f473 + Fix thinko in a92c6d77dcd475579c39bdff141f5eb128e2a048 + Include interface name in more errors printed by dhcp-discover + Check lock ownership only when debugging shared memory locks. This increases the general execution speed because getting PID and TID is a slow process. + Subtle change to priority of --server types. + Propagate dnsmasq defines into target FTL + Simplify FTL_iface() + Add pi.hole PTR record if requested IP matches the address of a local interface + Add config option PIHOLE_PTR to control the new auto-PTR behavior. + Do not reply with "pi.hole" to loopback PTRs + Add EDE return when no matching key found. + Add --quiet-tftp. + Fix forcing of reply type in regex replies only being done in debug mode (this never had any adverse effect) + Ensure shared memory is locked when reloading dnsmasq + Allow shorter IPv6 prefix lengths in (some) --synth-domain options. + --synth-domain now works in auth mode. + Return REFUSED in auth mode when we are not authoritative for the query. + Checks on prefix-length in --domain --synth-domain and --rev-server. + canonicalise_opt must always return heap memory. + Fix argument checking for --dhcp-match. + Detect malformed --dhcp-relay option. + Handle empty hostmaster in --auth-soa + Typo in new EDE code. + Add UINT32_MAX if not defined by system. + Add config option ADDR2LINE=true|false + Better fix than f2266d9678d71633d62d70238be3782ea74019c9 + Add additional checks for validity of data before trying to access it. Fixes #1151 + Properly handle edge-case when a query comes in at the exact end of the last overTime interval + Add further cache metrics + Warn about clients reaching rate-limit. Only warn once per interval and client to avoid log spamming. + Log for how many more seconds we rate-limit a client when this happens + Log rate-limiting of clients to the message table + Reload blockingmode on receipt of real-time signal 0 (a.k.a. pihole restartdns reload-lists) + Set extended DNS error to UNSET (-1) when importing from the database + Log how many queries have been saved in the final query storing + CONNTRACK needs CAP_NET_ADMIN. + Simplify linux capability check output + Fix NOERR/NXDOMAIN in answers configured by --domain-needed. + There was a `notify` variable to keep track whether a subscriber is observing our UBus object. However, it was not properly cleaned up in `ubus_destroy`, potentially becoming stale over UBus reconnections. The variable was removed and the current state is examined when sending notifications, similarly as is done in other existing OpenWrt code. + Re-order UBus teardown logic. + Remove remaining uses of deprecated inet_addr() function. + Remove remaining uses of deprecated inet_ntoa() + dhcp_buff2 not availble in log_packet, use daemon->addrbuff + Fiz sizeof() confusion in 527c3c7d0d3bb4bf5fad699f10cf0d1a45a54692 + Define order of reading files when --addn-hosts given a directory. + Revert "Re-order UBus teardown logic." + Revert "There was a `notify` variable to keep track whether a subscriber is" + Handle UBus serialization errors. + Eliminate redundant UBus `notify` variable. + Re-order UBus teardown logic. + Adjust logging levels for connmark patterns. + Make comment style consistent. + Use getnameinfo() instead of deprecated gethostbyaddr() for internal name resolving. + Log if hostname was imported from the network database. + Lookup IP addresses in local /etc/hosts file before sending out PTR requests + Allow users to configure how FTL reacts to queries when the gravity database is not available + Ensure we are not sending empty replies when we actually want to drop the entire answer + Ensure busy blocking is also done when database was not available initially (incl. when forking a TCP worker) + Log when adding entries to FTLs DNS cache (DEBUG_QUERIES) + Correct upstream->overTime when queries are blocked after they have already been forwarded upstream (e.g., during CNAME inspection) + Explicitly log when a retried query was a DNSSEC query. + Always count forwardings upstream, even if this was done for a (partially) cached CNAME + Remove redundant upstream->count + Some DEBUG_NETWORKING enhancements + Copy interface name before skipping when REPLY_ADDR is configured manually + Fix empty domain in server option parsing when more than one domain is given + Add BLOB reply type + Handle queries generated by FTL_make_answer() (i.e., blocked queries) as queries served from cache, not upstream (because they were never upstreamed) + Empty replies generated by FTL are NODATA (instead of BLOB) + Tests: DNS reply analysis test (using netmeister.org records) + Hard-code 8.8.8.8 as upstream server for the tests. It turned out to be more reliable as the CircleCI-provided DNS server tends to show a few timeouts on certain query types. + Tests: Use 1.1.1.1 as upstream as 8.8.8.8 SERVFAILs the HTTPS and SVCB tests domains + 1.1.1.1 rejects ANY queries... + Support limited wildcards in the input tags for --tag-if. + Rationalise query-reply logging. + Store validation result of internally generated DNSSEC queries + Store validation result of queries answered from cache + Avoid duplicated NXDOMAIN PTR queries. There is no no need to temporarily force FTL as system resolver when it is already the primary sytem resolver + Tests: Adjust for DNSSEC status now included for cache replies + Final logging tweaks. + Skip DNSSEC analysis if DNSSEC validation is disabled. Add new DEBUG_DNSSEC flag. + Tests: We want extra logging enabled in pihole.log during the tests + Tests: Never lauch DNS resolver thread when names are not to be resolved (e.g., on the CI) + Tests: Use pihole-FTL.pid when reloading to ensure the signal is not sent to a TCP worker (which would just ignore it altogether) + Tests: Use OpenDNS only for dig tests, use Google DNS for everything else. + Tests: Enable DNSSEC for query validation during the CI tests + Only open database when really necessary. This may reduce disk activity slightly and save a bit of CPU time. + Update DB counters still within the running TRANSACTION to reduce disk I/O + dhcp-discover: Implement Classless Static Route Option (options 121 and 249) + Get logging of DNSSEC status right when Checking Disabled bit set. + Add RFC 4833 DHCP options "posix-timezone" and "tzdb-timezone". + Prevent a possible deadlock in dhcp-discover. + Also check for capabilities CAP_IPC_LOCK and CAP_CHOWN + Tests: Adjust for newly added capability warnings. + Improvements suggested by cppcheck + Ensure we can the correct error string when "ip neigh show" or "ip address show" fails. Before, we picked up the error from the logg() which was likely always a not ver helpful "Success" message + Abort database routines early if database is known to be broken due to database file corruption. + Treat ANY queries the same as CNAME queries WRT to DNSSEC on CNAME targets. + Add regex extension ";reply=NXDOMAIN,NODATA,REFUSED,IP,NONE" + Tests: Add new regex extension tests + Implement support for custom redirection targets in regex extension, e.g., "someregex;reply=1.2.3.4;reply=fe80::1234" + Tests: Add tests for regex extension "reply=1.2.3.4", "reply=fe80:1234", and "reply=1.2.3.4;reply=fe80:1234" + Caching cleanup. Use cached NXDOMAIN to answer queries of any type. + Skip ascii-only names IDN processing + Revert "Skip ascii-only names IDN processing" + check_name() determines if IDN processing is needed. + Add all current RR types to the table of type names used for query logging. + Required FTL changes due to the preceding dnsmasq commit. + Small sanity check in wildcard tag matching code. + Retry on interrupted error in tftp + Add safety checks to places pointed by Coverity + Fix bunch of warnings in auth.c + Fix coverity formats issues in blockdata + Retry dhcp6 ping on interrupts + Fix coverity warnings on dbus + Address coverity issues detected in util.c + Fix coverity detected issues in option.c + Fix coverity detected issue in radv.c + Fix coverity detected issues in cache.c + Tests: "TYPE5" is now "[CNAME]" + Add NEG flag when replying to queries with forced NXDOMAIN. This ensures logging is correct and that the web interface will show the correct status. + Tests: Check Mozilla canary domain is blocked and logged correctly + Add PIHOLE_PTR=HOSTNAME allowing users to specify that Pi-hole should respond with the device's hostname (instead of "pi.hole") for local interface IP address PTR requests. + Valid option values for PIHOLE_PTR are now "PI.HOLE" (default), "HOSTNAME" or "NONE" + Add final newline + Trim excess whitespace + Add handling for "pi.hole.<local_domain>" and "<hostname>.<local_domain>". This fixes #1168 + Ensure virtual interfaces are recognized as distinct interfaces when finding their bound addresses + Reply with NODATA (instead of 0.0.0.0 or ::) if the interface we received a query on doesn't have the requested address type (e.g. virtual interfaces only configured with one IPv6 but no IPv6 address) + Fix coverity issues detected in domain-match.c + Fix coverity detected issues in dnsmasq.c + Fix coverity issues in dnssec.c + Fix confusion is server=/domain/# combined with server|address=/domain/.... + Add support for arbitrary prefix lengths in --rev-server and --domain=....,local + Thinko in immediately previous commit. + Optimize inserting records into server list. + Improvements based on static-analysis of source code + Fix --address=/#/...... which was lost in 2.86 + Correcly warn if dynamic directory is actually no directory + Make TTL served for blocked queries independent from local-tll setting in dnsmasq's config. + Improve last patch by splitting the previously combined if + Make --rebind-domain-ok work with IDN. + Change database permission to 664 + Set database permissions everytime the database is initialized + Change test suite to reflect changed file permissions + Fix indentation + Add special handling of iCloud Private Relay domains + Improve empty domain name handling + Add GitHub Actions integration + Add --nftset option, like --ipset but for the newer nftables. + Update embedded dnsmasq version to v2.87test2 + Tweak expected result for line 8 in "Get all queries shows expected content" + Ready GHA to take over from circle... + Fix a test that was already fixed, but then unfixed by a dodgy merge commit + Add in upload to our server + Single * is not enough it seems ------------------------------------------------------------------- Sat Sep 11 04:44:35 UTC 2021 - Samu Voutilainen <smar@smar.fi> - Added patch ftl-2.8.1-build-fix.patch. Fixes Tumbleweed build. - Miscellaneous fixes to spec. ------------------------------------------------------------------- Sun May 16 03:31:41 UTC 2021 - Samu Voutilainen <smar@smar.fi> - systemd service needs to clean up SHM files manually in order to avoid a failure in FTL restart. -------------------------------------------------------------------- Wed May 5 10:57:28 UTC 2021 - pihole-suse-packages@smar.fi - Update to version v5.8.1 + Retried queries due to missing DNSSEC valdiation have no upstream server (the related DNSSEC queries where retried, not this one). Hence, we shouldn't update the counts of any upstream here. This silences an incorrect "FATAL: Trying to access upstream ID -1" warning in the logs. + Do not terminate threads which may not be running. They'll be cleaned up at process termination anyway. + Ensure we clean up always behind us. Also when FTL crashes + Also clean up when crashing + Improve process-already-running detection + Tests: Update tests for new expected output on two concurrent instances + Terminate threads before closing database connections and finishing shared memory + Clean up after dnsmasq errors (port not available config errors, etc.) + Do not detach threads we want to be able to cancel and add logfile log to shared memory locks. Other forks may want to log as well. + Change to refreshed logo. + Give the images some space. + Center vortex. + Remove incorrect informaion. + Use dropshadowed logo + Escape DHCP options if necessary + Print raw bytes for unknown DHCP options + Implement DHCPv4 PCP Option (RFC 7291) + Resize shared memory only when locking. This ensures all shm pointers are invariant inside locks. + Preallocate one pagesize (usually 4K) for per-client-regex data. + Reduce code-duplication by using an array of shared memory pointers we can iterate on when chown-ing or deleteing. + Fix incorrect printf format identifier + Fix problem with DNS retries in 2.83/2.84. + Simplify preceding fix. + The preceeding commit changes the handling of retried queries. The logic is now changed so that distinct requests for repeated queries still get merged into a single ID/source port, but they now always trigger a re-try upstream. This effectively removes our IN-PROGRESS status so we remove the code handling this as well. + dhcp-host selection fix for v4/v6. + Correct occasional --bind-dynamic synchronization break + Always use <poll.h> + Move flags to recvmsg function in netlink + Obtain MTU of interface only when it would be used + Update embedded SQLite engine to 3.35.0 + Update .gitignore and add VSCode workspace exclude-settings + Add --dynamic-host option. + Add --log-debug option and MS_DEBUG flag to my_syslog(). + Only log changes to DNS listeners when --log-debug is set. + Log creation of listeners and enable dnsmasq log-debug when any FTL debug option is set. + Fix a memory leak when re-opening the databases (when forking or reloading the lists). The memory leak is on the order of a few bytes but scales quickly with the number of clients. It is caused by SQLite3 not being able to clean up behind itself when we're not finalizing and closing everything explicitly. + Avoid jump depending on uninitialized bytes (only relevant in debug mode). + Join canceled threads on exit to ensure they exited properly before we exit from the main process. This includes waiting for them to clean up their own stack memory, etc. + Ensure we close FTL database connection when exiting the main process. This has no consequences else than silencing some meomry-lost complaints by valgrind (any allocated memory is release on process exit anyway) + Ensure shared memory strings bucket is large enough when locking. Do not resize it when we are holding the lock. Also, optimize FTL-domains size + Don't try to finalize gravity statements two times + More fine-grained locking in network table processing should decrease delays in DNS resolution on very slow machines + Reduce rate-limiting checking to once per second (rather than every 100 msec) + Simplify locking during network table processing and generalize spacial handling for virtual interfaces (hwaddr 00:00:00:00:00:00) + Simplify signal handling and catch SIGABRT in addition + tftp warning fix. + Teach --bogus-nxdomain and --ignore-address to take a subnet argument. + Use random source ports where possible if source addresses/interfaces in use. + Update SQLite3 from 3.35.0 to 3.35.2 + Do not skip remapping if the size hasn't changed + Avoid leaking memory if dbquery() fails + Automatically reply with IP address a query came in from when in blockingmode=IP + Scan through local interfaces to find IPv4/IPv6 addresses to reply with in IP blocking mode + Add fallback in case docker does not reveal the interface we're running in + Simplify and unify interface address derivation + Do not close FTL database connection when forking TCP workers + Open database after forking + Add timeout to joining of threads + Remove additional log file locking + Open individual database connections where we need them. Do not use global pointers anywhere. This may mean we have more than one connection open at the sae time. SQLite3 will take care of thread-safety. + Fix FTBS on FreeBSD due to Linux-specific optimisation of if_nametoindex() + Always set database pointer to NULL, even when closing failed + Prepare for dnsmasq code refactoring patches. This commit needs to be undone later. + Reduce few repetitions in forward code + Create common function for forward dump, log and send + Move repeated test pattern to server_test_type + If the first argument ends in ".lua", we immediately start the embedded LUA engine. Same for ".db" and ".sql" files which are directly routed into the embedded SQLite3 engine. + Add tests for new feature + Favor ULA and GUA addresses over LL when picking an IP address for replying to blocked AAAA queries. + MUSL and GNU C define the substructure of in6_addr differently so we cannot rely on being able to access the substructure directly. + Use properly-sized buffer for format_time() + Fix thinko in 51f7bc924cbcdeb09cbb83249b70c121d1ffa31e + Change the method of allocation of random source ports for DNS. + Scale the DNS random scket pool on the value of dns-forward-max. + Update SQLite3 from 3.35.2 to 3.35.3 + Ensure FTL can be compiled from source archives offered by GitHub for each release + Print special notice when no version can be obtained + Improve error reporting in network table routines + Also log ignored extra regex extensions to the message database table + Prevent forks from adding regex compilation errors to the message table + mpid() should return PID even if we are not forking at all + Log correct database index on regex warnings + Correct missing SERV_DO_DNSSEC flag, add new spot + Enable DNSSEC compilation on nettle 2.7.1 + Replace ad-hoc libnettle version detecion with MIN_VERSION macro. + Fix spacing in translatable strings. + Re-add FTL hooks into dnsmasq's forward code + Update dnsmasq version string to 2.85 + Circle CI: skip uploading build artifacts on forks + TFTP tweak. + Update SQLite3 from 3.35.3 to 3.35.4 + Do not flag query as retried when we decide ourselves that it should be retried without any new query triggering this. Deprecate DEBUG_EXTBLOCKED (now covered by DEBUG_QUERIES and add DEBUG_STATUS) + Ignore duplicated replies to the same query. This is useful in general and also happens to circumvent a dnsmasq bug (we already reported this one upstream). + Subtly change behaviour on repeated DNS query. + Simplify status and reply type handling in FTL + Ensure we always set the status of cached queries + Assert size of countersStruct + Combine queries for the same DNS name if close in time. + Handle resource exhaustion of struct frec_src same as struct frec. + Ensure reply type is always stored for cached queries + Re-add IN_PROGRESS query status + Do not try to log if no log file is defined + Prevent a possible infitite loop in the inunterruptible syscalls. + Queries read from the database need to be counted as unknown before restoring the query status + Add missing newline after "Notice: Found no readable FTL config file" + Add config options REPLY_ADDR4 and REPLY_ADDR6 to overwrite automatic IP detection in IP blocking mode. + Use MAXLOGAGE to control which queries get deleted by GC + Tidy error logging in 961daf8f921503457d1f539f79b3a2def7d479e2 + Work around warning on tag build due to && logic. + Fix database update to version 7 reporting error when there is none. This is not a critical bug as the issue resolves itself on the next start of FTL. + Test: Add test for "database not available" messages indicating failed database updates and creations. + Give threads a bit more time to reach a point where cancellation is safe. We cannot give them too much time because, otherwise, the proces trying to TERMinate FTL may decide to KILL it instead. We should avoid this to be able to properly cleanup. + Don't try to terminate threads when we never launched them. -------------------------------------------------------------------- Wed Apr 14 11:04:39 UTC 2021 - pihole-suse-packages@smar.fi - Update to version v5.7 + Fix incorrect "FATAL" error message during garbage collection + Fix incorrect "FATAL" error message during garbage collection + Move fd into frec_src, fixes 15b60ddf935a531269bb8c68198de012a4967156 + Fix to 75e2f0aec33e58ef5b8d4d107d821c215a52827c + Optimise sort_rrset for the case where the RR type no canonicalisation. + Fix for 12af2b171de0d678d98583e2190789e544440e02 + Don't display unrelated CNAME queries when filtering for specific domain + dnsmasq-v2.83 forwards multiple queries to the same destination only once and stores the other queries as duplicates. They do receive the answer later on, however, this is usually not logged (when log-queries=extra is enabled, there will be a warning about the duplicate). This commit handles such duplicates and introduces a new reply type 14 = "already forwarded" + When seeing duplicated queries, the original query may have been blocked during CNAME inspection. In this case, we need to change the status from "OK (already forwarded)" to the correspondig blocked status. The "already forwarded" information is lost but that seems okay. + Check source query for its status when checking if we need to update the duplicated ones + Tidy initialisation in hash_questions.c + Change HAVE_NETTLEHASH compile-time to HAVE_CRYPTOHASH. + Bump copyright notices for 2021. Happy New Year! + Fix possible free-memory ref in e75069f79aa6b8a61034a9a4db9b6265b8be8ae4 + Fixes incorrect "Found unknown status 14 in long term database" warning in the logs. We change the code to use a enum-based struct so we cannot forget to update this in the future when adding further query status types. + Add per-client rate-limiting. The default limit is 1000 queries in 60 seconds. + Add output of how much memory in /dev/shm is used by FTL itself + Try to create shared memory objects before reading the settings + Do not try to delete existing shmem objects on start - that may cause running FTL instances to crash when it tries to access them. Instead, new instances should properly fail to start. + Tests: Running a second instance is detected and prevented, FTL continues to work as expected afterwards + Do not explicitly request a lease time in our DHCPREQUEST as this may lead to incorrect responses. Also, when sending a request to lo, we should send it to the interface address instead of the broadcast (lo doesn't support broadcast destinations). + Fix queries sent upstream being counted incorrectly when modified later on (blocked externally, blocked during CNAME inspection). This also applies to queries loaded from the database. + Increment forward counters when immporting QUERY_RETRIED or QUERY_RETRIED_DNSSEC fromthe database + Retain EDNS0 bits from incoming queries when blocking requests -------------------------------------------------------------------- Tue Jan 19 14:43:45 UTC 2021 - pihole-suse-packages@smar.fi - Update to version v5.5 + Detect and handle interface changes of clients with the same IP + Update SQLite3 to 3.34.0 and expose sqlite3 shell as 'pihole-FTL sqlite3' (drop-in replacement is available as well) + mend + Added missing NS query type to getQueryTypes() + Log date/time of FTL in header just as SQLite3 does as well + Test for embedded SQLite3 shell available and functional + Modified test for NS type + Fix for errno not being set by posix_fallocate() in contrast to fallocte() who did set it. + Add new query types SVCB and HTTPS + Tests: Add SVCB and HTTPS as expected query types + Implement support for displaying exact type instead of the catch-them-all category OTHER. The OTHER category is still used when it comes to computing statistics to ensure your chart's legend does not explode. + We cannot really decide whether local configuration lines are meant for blocking or something else. Just record such queries as replied to from cache because this is what they are. This code made sense at the time where wildcards were implemented as dnsmasq config lines, however, we've advanced to our own regex engine since then and all config lines should have also been auto-migrated. + Clarify comment + Only return regex index when allowed by privacy settings. This may leak information, otherwise. + Check for validity if iface pointe before dereferencing it. + Don't show retried queries when filtering for blocked queries. + Optimize datastructures using bitfields and item re-arrangement (to minimize padding). This reduces the size of query, client, and regex records by 8 bytes per item. Note that this optimization was done on x86_64 and may not apply for other architectures (32bit architectures already used less padding). + Statically assert struct sizes are what we expect. This prevents us from increasing the memory needs unintentionally (e.g. due to sub-optimal padding) + Store blocked property in query flags. + Use blocked property in API code. Make query->upstreamID = -1 the new default to differentiate easily what was forwarded (ID will be >= 0) and what not (ID == -1). Store the upstream server also for other query types that were forwarded (like queries blocked during CNAME inspection). + Add MAXDBDAYS=-1 to disable auto-cleaning and ensure overflow cannot happen (we just enforce the maximum in this case) + pxe: support pxe clients with custom vendor-class + Use the values of --min-port and --max-port in TCP connections. + Fix remote buffer overflow CERT VU#434904 + Check destination of DNS UDP query replies. + Use SHA-256 to provide security against DNS cache poisoning. + Optimse RR digest calculation in DNSSEC. + Fix DNS reply when asking for DNSSEC and a validated CNAME is already cached. + Add missing check for NULL return from allocate_rfd(). + Handle multiple identical near simultaneous DNS queries better. + Handle caching with EDNS options better. + Support hash function from nettle (only) + Small cleanups in frec_src datastucture handling. + Adapt for change in struct forward to forward->frec_src + Update dnsmasq version string + Fix warning message logic. + Update to new struct frec fields in conntrack code. -------------------------------------------------------------------- Tue Jan 12 05:14:26 UTC 2021 - pihole-suse-packages@smar.fi - Update to version v5.3.4 + Show BOOTP server and file strings used by TFTP + Update dnsmasq version to 2.82 + Use fork-private regex substructure because each regex has an opaque structure (once compiled) and cannot be kept globally available through shared memory (at least not with any realistic effort) + We have to explicitly set conflinebuffersize to zero when freeing the buffer itself to avoid getline() crashing in some special edge-cases + Rename memory.c -> syscalls.c + Factor out syscalls for calloc, free, realloc and strdup into dedicated syscalls/{}.c files + Add interrupt-safe fprintf() and printf() routines + Add interrupt-safe vfprintf() and vprintf() routines + Make calloc(), realloc() and strdup() interrupt-safe + Add interrupt-safe write() routine + Add interrupt-safe accept() routine + Avoid redundant error reporting + Improve printf(), fprintf(), vprintf(), and vfprintf() error reporting + Add interrupt-safe recv() routine + Add interrupt-safe recvfrom() routine + Add interrupt-safe pthread_mutex_lock() routine + Add interrupt-safe select() routine + Add interrupt-safe fopen() routine + Add interrupt-safe sendto() routine + Backup and restore errno in real-time signal handler. + Add interrupt-safe vsnprintf() routine + Add interrupt-safe snprintf() routine + Add interrupt-safe vsprintf() routine + Add interrupt-safe sprintf() routine + Show complete list of args when complaining about unsupported argument + Adjust test for unknown argument to support the new format + Expose lua-interpreter as virtual pihole-lua binary + Add drop-in support for lua binary + Add drop-in replacement support for luac as well + Fix freeing regex pointers to set the global not the local object to NULL after free(). + Add interrupt-safe asprintf() and vasprintf() routines + Add more debugging output for domain reloading (on receipt of SIGHUP) + Add REFRESH_HOSTNAMES=UNKNWON to support only refreshing recently active clients with unknown hostnames + Force refreshing of hostnames (according to REFRESH_HOSTNAMES config) on receipt of SIGRT4 + Give explicit reason for skipping in debug message + Fall back to using ftruncate() when fallocate() return with "Operation not supported". This may happen if the kernel is older than 2.6.23 or glibc older than 2.10. ftruncate() has its own disadvantages, however, it is POSIX compliant (POSIX.1-2001) so should be supported even by ancient kernels. + Add new DEBUG_EXTRA flag used for special (temporary) debugging + Update src/resolve.c + num_regex is not in counters any more + Enable extra logging only when DEBUG_EXTRA is set + docs: fix simple typo, timestemp -> timestamp + Add interrupt-safe fallocate() routine, due to the special nature of the fallocate() macro, we hav to use a modified name fTLallocate() to implement this function + Prevent possible deadlock if log is not writable (e.g., permission denied) + Don't fail when trying to free(NULL) + Fix Unix socket error handling + Do not print user change information if there is no user change + Reply with configured BLOCKINGMODE to blocked CNAME requests + Revert "Improve compatibility with old (ancient) kernels" + Analyze original question and use it to decide whether we mock an A or AAAA reply when blocking + Don't iterate over all clients every minute trying to find new ones but only do this when the RESOLVE_NEW_HOSTNAMES event is set + Add DEBUG_EXTRA flag (#994) + Escape spaces by ~ + Do not sync after executing regular expression on a domain - Use proper version handling in spec -------------------------------------------------------------------- Sat Dec 5 07:02:17 UTC 2020 - pihole-suse-packages@smar.fi - Refreshed patch shared_libraries.patch - gmp needs to be statically linked for Tumbleweed - Update to version v5.3.2 + Add additional_info column to test database (query table) + All queries: Hide UNKNOWN queries when not requesting both query status types + Add ability to connect to shared memory of the running FTL process + Bundle lua library "inspect" + Automatically load bundled libraries and make them available globally. + Test: Automatically loaded libraries + Add pihole.query([idx]) + Test: pihole.query(0) returns details of the first query + Include FTL's prototypes in LUA + Remove shm data sourcing. We will query such data through the API with Pi-hole v6.0 + Add support for ECS subnet parsing in FTL. + Log previously seen client when interpreting EDNS0 client subnet information + Evaluate possible EDNS data before analyzing a new query + Analyse ECS information only if EDNS0_ECS is enabled (enabled by default) + Add support for EDNS(0) CPE-ID (Common Platform Enumeration Identifier) + Protect against possible buffer overflow due to a malicious/malformed EDNS(0) payload + Add support for EDNS(0) MAC in BYTE format (dnsmasq option add-mac) + Add support for EDNS(0) MAC in TEXT format (dnsmasq option add-mac=text) + Add partial support for EDNS(0) MAC in BASE64 format (dnsmasq option add-mac=base64) + Correct name is EDNS(0) not EDNS0. + EDNS(0) debug message fine-tuning + Use preprocessor constants for OPTCODES to improve readability of the code + Add partial support for EDNS(0) COOKIES + Tests: EDNS(0) analysis + Make EDNS MAC available for FTL_new_query() + Tests: Simplify EDNS(0) tests + Use %z to print size_t for both 32 and 64 bit compatibility + The version of dig in the CI containers is too old for the option +cookie. Simulate the same with +ednsopt + Improve regex engine. This adds new features such as in-code comments, approximate matching (fuzzy matching) + Add regex-test mode + Allow test-regex mode to be started without log ans shared memory (alleviates write permission issues when running pihole-FTL as a different user) + Reduce overall costs by not always calling the approximate matching algorithm. + Add 26 regex tests (following https://discourse.pi-hole.net/t/regex-engine-improvements/34751) + Make regex-text output more user-friendly + Tests: Bats ignored empty lines + Mark all the new cli_{}() functions as ((const)) to make them subject to common subexpression elimination. + Use info box for step reporting + Adjust empty lines + Tests: Add tests for useful error hints for incorrect regex. + Mark get_regex_from_rowid as pure. + Modify regex-test mode for better batch-processing capabilities + Add quiet regex test mode for inclusion in pihole -q + Make quiet mode really quiet. However, speak up when there are regex errors + Tests: Test quiet regex-test mode + Simplify memory structure of regular expressions inside FTL. This allows for future regex extensions. + Add ;querytype=AAAA option + Case-insensitive query type checking in regex extra instructions + Add ;invert option + Make querytype string available everywhere in FTL. + Add ;querytype=!A option for INVERTED query type filtering. + Add tests for ";querytype=A", ";querytype=!A" and ";invert". Add explicit support for query type NS. + Terminate running FTL instance (if any) before starting tests. + Log invalid querytype as warning to the Pi-hole diagnosis system + Only print time/ID string when not in direct user interaction (CLI mode) + Fix API computation error introduced in e0609f14eee7903bca93020371576dad0ca93338 + Blocking PTR requests may have been done unintentionally, print a warning about this + Warn if specified more than one querytype option (the last one wins) + Add tests for new ;querytype sanity check warnings. + Undo PTR blocking warning + Try to obtain MAC address from dnsmasq's cache (also ask the kernel) instead of only relying on the database content (may not be fully up-to-date) + Tests: Need to test also for the interface being specified in the log + Remove left-over debugging output + Tests: Tweak test to recognize new debug output format + Implement super-client infrastructure + Try to obtain MAC address from dnsmasq's cache (also ask the kernel) instead of only relying on the database content (may not be fully up-to-date) + Read superclients from new FTL database table + Import super-clients during start (before all other clients are added) + Implement client-based Query Log filtering for super-clients and ensure we always count both the normal and the super-client when there is a new query / garbage collection + Rename table from "superclients" to "superclient". + Tests: Database has been updated to version 9 + Change concept of super-clients from MAC-based to index-based. We append a new column to the network table which can be used to assign super-clients to multiple devices. This can both cover automatic grouping (whenever MAC addresses are available) and also situations where this is not possible (when MAC addresses are not available, e.g., due to network layer separation). Real-time signal 3 causes FTL to re-import super-clients from the database without affecting anything else. + Move signal handling into a thread to avoid possible lock race-collisions + Tests: Test for correct import and assignment of super-client + Ensure we cannot end up in a self-locking state when opening the database. + Explicitly case time_t to (long long) before printing to address the musl-decision to make time_t 64 bit on 32 bit machines + Show also possible IPv6 nameservers. They are stored in an auxiliary (external) structure so they were not included in the debug outputs (even when they were used) + Use EDNS(0) MAC address for the network table (if available) + Do not try to locate a previously used mock device when EDNS(0) MAC data is available. + Add more debugging output to network table processing + Ensure mock-devices which are not assigned to any addresses any more (they have been converted to "real" devices), are removed at this point + Do not re-open gravity database when not forking for TCP workers (debug mode) and simplify network table routines (remove code duplication and prevent possible dead-locks when trying to resolve host names) + A small fix ensuring that we can determing the interface a query came in for all clients (also localhost) + Unify network table debug messages + Parse the kernel's Internet protocol address management to get information about local interfaces. + Delete addresses from network_addresses table which have not be seen for 7 days + Only try to resolve host names of upstream servers which were recently active. The current limit for "recently active" is hard-coded to two hours. + Fix error displaying if the upstream server replied with REFUSED or SERVFAIL + Extend domain filtering to also check the CNAME domain for domain-filtering (if this is indicated by the query status) + Improve API filtering for domains + Try to obtain host names from another address of the same device when there is none for the exact address (may happen, e.g., for IPv6 addresses) + Use already existing (but by default disabled) cleaning. The interval is customizable and defaults to MAXDBDAYS. + Fix syntax error + Remove duplicate function getDatabaseHostname() + Add explicit event queue to avoid possible race collisions when many signals arrive at the same time (or very very close to each other) + Fix subdirectory include paths. This is only to be explicit, the relative search finds them otherwise as well + Signals are not handled asynchroneously. Add additional delays in the tests to avoid them failing due to aksing too early for a result. + Add new dhcp-discover command + Implement multi-threaded scanning (constant scanning time regardless of the number of interfaces) + Increase timout to 10 seconds and ensure logging cannot be interrupted (for readability) + Be a bit more specific about binding errors + Also send DHCPREQUEST on unconfigured interfaces + Use unsigned 32bit variable for the XID everywhere + Implement DHCP options 44 and 252 (non-standard WPAD extension). Improve human-readable time formatter. + Do not print WPAD path if it is a control squence. + Fix GCC9 regression for printing the same buffer into itself in sprintf() + Request a lease with validity of 1 second in the DHCPDISCOVER packet + Do not try to free NULL pointer in resolveAndAddHostname + Show debug messages only in debug mode + Do not block shared memory when inactive clients are skipped. + Upload FTL log to tricorder.pi-hole.net instead of printing directly into the container output + Add more debug logging to getDatabaseHostname() + Do not skip recently inactive clients in ARP/neighbor table processing as they may still need properties to be updated (like host names, etc.) + Add real-time signal 4 to re-resolve all host names (clients + upstream servers) + Add real-time signal 5 to request ARP/neighbor parsing + Reset actions after the threads picked up the new real-time signals + Handling of clients not in ARP has been moved into add_FTL_clients_to_network_table() + Add more verbose version output (./pihole-FTL -vv) + Do not block shared memory when inactive upstreams are skipped. This was missed in #889 + Rename resolveForwardDestinations() to resolveUpstreams() and make private functions static. + Every time FTL allocates more memory, we explicitly log how much (out of how much) space is used in /dev/shm + Explicitly warn users if space tends to be running out in /dev/shm + Handle SIGBUS, SIGILL and SIGFPE events in our crash reporter. Give human-readable explanations of why this happened where possible. + Only use stsvfs data is the function returned no error. + Explicitly cast the block counts to unsigned long long to avoid overflowing with drives larger than 4 GB on 32bit systems + Reopening the FTL database may lead to rare race-collisions in SQLite3. We avoid them by keeping the database connection open all the time. + Open database for history-reading + Improve checking boundaries of the shm_per_client_regex shmem object + Make realloc_shm message more informative + Add more comments to the code + Ensure to remap the per-client-regex struct when it is changed in a fork. + Use posix_fallocate() instead of ftruncate() when resizing and/or creating shared memory objects. This ensures we reserve the requested memory exclusively for ourselves. + Exit immediately if fatal memory errors happen + Add new status RETRIED (12) to be used for queries which were retried. If a query was retried five times before it suceeded, queries 1-4 will be marked as RETRIED and only query 5 will stay in status FORWARDED. + Use new armv4, armv5, armv6hf, armv7hf containers to build the corresponding binaries + Also handle retry events when the retry happened in the small timeframe of when we already have the upstream response but DNSSEC validation is still ongoing + Retried DNSSEC queries are ignored, we have to flag themselves. Retried normal queries take over, we have to flat the original query. + Move call to resolveNetworkTableNames() from resolver into database thread + Make timer output at termination of FTL human readable (days/hours/minutes/seconds). + Mark database as being available when creating a new database to avoid FTL skipping adding the tables thinking the database connection isn't ready. + Ignore ECS loopback addresses to avoid rewriting the client IP to a (useless because distant) localhost + Improve query interface origin determination + Add tests for ECS loopback ignoring + Upload binaries into writable html-subdirectory and download+verify uploaded binaries in an additional CI step. + Add new DEBUG_HELPER option. It logs any helper activity (and possible errors) to pihole-FTL.log + CMake install: update setcap to add the CAP_SYS_NICE capability + Log information about the user FTL is running as and if we're dropping to another user (such as nobody/nogroup) + Include upstream details in all-queries API response + Make FTL upstream destination port-aware + Rename super-clients ---> alias-clients + Catch fatal dnsmasq errors caused by incorrect config lines and print it in pihole-FTL.log + Store fatal dnsmasq message in message table so it can be used by the Pi-hole dashboard diagnosis system. + Add attribute gnu_printf to the new function FTL_log_dnsmasq_fatal() + Define default script path in /opt/pihole/libs and always build READLINE support if static libraries are available on the system + Download and install LUA script during test runs + Only print history debug messages when in debug mode + Update LUA 5.4.0 -> 5.4.1 + Print hint that readline isn't available only in debug output. Otherwise, this output might leak into script executions when readline support is not compiled into the binary (missing libraries at build time). + Add API callback to remove DHCP leases without the need for a restart of the DNS/DHCP server + Add API debug messages + Skip clients with no active counts at all (may be old IPv6 addresses) + Keep upper case characters in host names because they may make them more readable (like FritzBox, WDMyCloud, or VacuumRobot) + Clarify that disabling the database only disables storing queries in the database. We still use the database for storing messages (such as regex syntax warnings) and alias-clients. + Silence warning about copying a NULL pointer for DHCP clients without a hostname + Print hint when database query importing is enabled but exporting is disabled - this may not what the user want. + Analyze all DHCP options dnsmasq is aware of + Convert numbers from net to host order before displaying them + Implement special handling for "pihole-FTL -- --help dhcp" and "pihole-FTL -- --help dhcp6" + Remove buster-specific binary test output + Downgrade expected glibc version and expect that stretch does not build a v5TE binary explicitly (instead, it does v4T) + Use new stretch-based ftl-build:v1.8 containers + Fix compatibility with GCC 10. + Add -fno-common to HARDENING_FLAGS. + Do not warn about query status 12 and 13 on import (retried queries) + Catch all real-time signals, decide later which one we handle and which one we ignore. + Check for memory allocation erros in parse_FTLconf() + Respect settings RESOLVE_IPV4 and RESOLVE_IPv6 also when trying to resolve host names from the database (network table) + Do not try to resolve IPs for records without hostnames in the network_addresses table. + Add new REFRESH_HOSTNAMES option + Use case-insensitive comparison of MAC address to ensure capitilization does not play a role. + Tweak code to restore compatibility with Gentoo gcc 10.2.0-r3 + Always try to resolve hostnames when seeing a client/upstream serer for the first time. Also when it wasn't recently active (may happen on re-import from database history). + Add more debugging output and ensure refreshing rules are really only used when refreshing -------------------------------------------------------------------- Tue Sep 15 07:40:28 UTC 2020 - pihole-suse-packages@smar.fi - Update to version v5.2 + Move counters definition from memory.c to shmem.c magically clears a lot of (wrong) VSCode errors. Doing this on request of a user as it is harmless. + Import unknown clients from ARP table + Explicitly set prepared statements to NULL when they are finalized. + Explicitly log if we had to make assumptions because the gravity database was not available. + Add DELAY_STARTUP setting to delay startup of the embedded dnsmasq. + Remove option FORCE_LOCAL_RESOLVER as we do not need it. + Add more comments, only print debugging output when DEBUG_DATABASE is enabled. + Simplify SQLite 3 database extension + Convert recently found (at most 1 hour old) mock-devices into "real" when we gather ARP/neigh information about them. + Fix nameserver list in auth mode. + Allow overriding of ubus service name. + CircleCI has an unforseeable number of devices in its ARP cache. Do not check for a strict number of clients during the tests. No changes to the source code. + Ensure blocking also works when the long-term database is not used. This was broken before as we returned too early (the SQLite3 engine was not yet fully initialized) when the long-term database was disabled. + Fix possible memory leak in config.c + Some general tweaks + Explicitly log failures in creating the new sqlite3 function. + Ensure we don't loose memory after ARP cache parsing. + Also return NO MATCH when invoking subnet_match() with non-TEXT arguments. + Add a comment that gethostbyaddr() may leak memory (only once, not seen leakage of more than 110 bytes) + Check arguments are of type SQLITE3_TEXT + Initialize resolver subroutines if trying to resolve for the first time + Only check/set client status when size of the array is not exceeded. Skip otherwise. + Do not import unknown clients from the ARP cache into FTL's memory. It is not our job to care about them if they are not doing any DNS queries. + Ensure ARP strings are NULL-terminated + Exiting instead of aborting may be benefitial in FTL forks. + Print arguments passed to embedded dnsmasq when at least one DEBUG flag is set. + Re-open gravity database (and re-prepare database statements) before accessing the database in case FTL forked. + Memorize PID of this thread to avoid re-opening the gravity database connection multiple times for the same fork + Implement process-private prepared gravity database client statements. This fixes an incompatibility across forks when serving TCP traffic using dedicated workers. + Silently increase size of vector if trying to read out-of-bounds + Explicitly include type definition of int16_t in config.h as needed by the musl-compiler + Remove append and delete instructions as we will always identify clients exactly by their IDs + Musl's realloc() does not zero any memory. Do this manually. + Free allocated memory after ordinary termination of TCP workers (TCP connection closed) + Fix rare problem allocating frec for DNSSEC. + Tweak to DNSSEC logging. + Restored astrisk match for auditlog + Correct, indent and simplify wildcard-compatible auditlog SQL logic. + Modify FTL's internal resolver to work in two phases: First, try to obtain a host name by using the internal resolver (i.e., FTL). In a second step, when FTL didn't know the answer, ask the resolvers as configured by resolv.conf. We've seen that the latter is necessary to get proper name resolution in docker environments. + Convert port from host to network byte order + src/dnsmasq/dnsmasq.c: Labeled a lonely #endif + Update dnsmasq version to pi-hole-v2.81 + Don't try setsockopt of non-existing NETLINK_NO_ENOBUFS option (fixes qemu issue). + Revert "Don't try setsockopt of non-existing NETLINK_NO_ENOBUFS option (fixes qemu issue)." + Convert failure of setsockopt(..., SOL_NETLINK, NETLINK_NO_ENOBUFS, ...) into warning. + Make regex matching case-insensitive by default and remove config option to control this. + Automatically block _esni.* subdomains of blocked domains. This can be disabled by setting BLOCK_ESNI=false in pihole-FTL.conf + Simplify blocking metadata forcing code. + Add full drop-in replacement mode pihole-FTL can use to mimic the dnsmasq binary. + Add a shortcut for dnsmasq syntax test + Do not decide whether we are blocking or not based on the gravity count (pre-v5.0 measure) but use the dedicated blockingstatus variable. + Use /run instead of /var/run for FTL runtime files + Deleted Swag store affiliate link + Invoking free_sqlite3_stmt_vec() on a NULL pointer should be a harmless no-op. + Check for validity of prepared statements before trying to use their get property. + Create "message" table and log regex errors in there. + Fix bit-order in subnet mask generation. + Bump build container version. + Use BLOBs as datatype for the custom columns to keep this feature as generic as possible. We can always append more columns to the end of the table whenever needed. + Store message type as string instead of enum values. + Ensure message table is also flushed on receipt of real-time signals. + Always chose most suitable (= maximum) subnet for clients. This allows to configure specific settings for a whole range of devices but still exclude others. Complain softly (no error) if multiple configured subnets match with the same number of relevant bits. + Ensure to finalize statement before closing the database connection. + Explicitly include limits.h in src/files.c to improve ppc64le support on Alpine. This fixes #751 + Store subnet warnings in the message table. + Add User-Agent to macvendor.py + Process cached SRV records + Simple optimizations + Skip CircleCI Upload step on foreign PRs. + Add addr2line output into our self-generated backtraces. + Only compile print_addr2line() when we can actually generate backtraces. + Add comment + Remove unneeded inet_ntop + Fix indentation + Fix indentation + Reload the privacy level when reloading the lists + Bump to v1.2 build images. + Add NAMES_FROM_NETDB option. + Store client group information in shared memory. + Add checks for the compiled binary for all supported CI platforms. + Don't need to call inet_ntop here + Also check result of "file pihole-FTL" to do some further checks on the generated binary. This now includes a check for the minimum supported Linux version. + Add cmake build + Implement install step to be the same as the Makefile build + Default install prefix to /usr + Put runtime output, i.e. pihole-FTL, in the root of the build dir + Move sqlite3-ext to the database target + Fix build of sqlite3. Move sqlite3 defines to the top level so they are on all. + Add "not stripped" to arm-qemu test + Reduce README.md + Fix static build + Really, really fix static build + Add boolean to be able to store if we decided which groups to be used (an empty string can actually mean no groups as a special case) + Fix broken RESOLVE_IPV{4,6} setting. + Fix possible memory issue by obtaining pointer only when it is guaranteed that the pointer will not change. + Remove Makefile + Add license headers to CMake files + Use cmake on CI to generate FTL binaries. + Lower needed cmake version to 2.8.12 + Add CI build script. + Add a build script for users. FTL can be build by simply running "./build.sh" + Use cmake ENV{} to actually aquire the env variables + Clarify comment + Also check REVOLCE_IPV{4,6} setting when trying to derive a host name from the FTL database. + Improve build.sh script. Add "install" and "clean" targets. Also ensure successive builds are possible to speed up the entire process. + Allow no/false and yes/true for all config options. + Re-aquire client and upstream pointers after a name resolution. As we're leaving the locked area for the resolve, we cannot control if the shared memory object changed meanwhile. If it did, then the pointers will point into nowhere, leading to a SEGV_MAPERR. + Set nice value of pihole-FTL (configurable) to increase DNS server performance. + Add CAP_SYS_NICE for the tests + Ignore missing CAP_SYS_NICE in the CI tests as we are not allowed to change the nicencess. + Add warning for invalid hostnames to FTL message table. + Only open FTL database for storing a message when there is not already an open connection. + Clarify warning that the check found AT LEAST one invalid character. + Ensure host name errors do not accumulate. + Close database on any erros to ensure nothing stays locked. + Do not listen to real-time signals in helper processes + Ignore real-time signals outside of the main process (such as in TCP forks) + Fix #805. This fixes a buffer overflow when handling TCP requests. Details are on the dnsmasq mailing list. + Ensure main process is terminated orderly when a fork fails miserably. + Send real-time signal 2 from forks to main process to signal it should terminate with EXIT_FAILURE. + Improve logging for forks and thread by including further details to the log ID + Include thread names in crash reports to ease debugging. + Print at least addresses when the addrline conversion wasn't successful. + Bind to socket in thread instead of main process to ensure forks do not inherit sockets they shouldn't. + Show fork created/terminated if any debug mode is enabled. Log reason for TCP worker termination (either client disconnected or connection timeout). + Remove portfile not used by the current web interface any longer. + Add support for additional query types: "A", "AAAA", "ANY", "SRV", "SOA", "PTR", "TXT", "NAPTR", "MX", "DS", "RRSIG", "DNSKEY" and "OTHER" (summing up all other possible DNS types) + Tests: Test for new query types + Don't use fshort-enums as it may make objects files incompatible. + Always re-open gravity database handle when forking + Children inherit file descriptors from their parents. As we don't need the API sockets in the forks, we clean them up after forking + Thorough clean-up following 8270648da1eae77db381b848a47d79b85c206e29. + Be explicit about if we count or query the number of domains. This simplifes the logic and makes it more readable. + Do not assume a query to google.com returns only one fixed AAAA record. + Remove privacy level 4. Systems currently running level 4 will automatically pick up the highest available level (which is now 3). Also tidy up enums into a dedicated file. + Add additional_info column to queries table. We fill it with the domain that caused blocking the entire CNAME chain. + Log CNAME blocking to pihole.log + Include new column in database schema test + Load domain causing the blocking in a CNAME inspection from the database during import. This ensures restarting FTL does not mean we lost this information. + Store + import ID of regex used for blocking in additional_info field + Fix an edge-case where CNAME blocking can be foiled when parts of the CNAME chain are already in the cache + Fix incorrect attribution of the blocked status to the wrong domain. This also simplifies the terminology in the CNAME inspection routine. + Revert "Remove portfile not used by the current web interface any longer." + Tweak revert commit. We should add the port after configuring it and independtly from opening IPv4 and/or IPv6 sockets. Also, we cannot delete the port in close_telnet_port() as this function is called by TCP workers since v5.1 so we'd loose the port file when the first TCP query comes in. + Test: Check port file exists and contains the expected number (4711) + Skip second termiantion if there is already a termination event in progress. This has been observed with clients clsoing their connection exactly at the same time when dnsmasq wants to close the connection itself due to a timeout (it thinks this client is stale). + Include the information for which client we have forked. Examplary message: "TCP worker forked for client 192.168.0.42 on interface enp0s25 (192.168.0.12)" + Double time until TCP worker timeout from 150 seconds to 300 seconds. RFC 1035 says: "If the server needs to close a dormant connection to reclaim resources, it should wait until the connection has been idle for a period on the order of two minutes." We are unlikely to run into a limit here as the total number of allowed TCP workers is fixed as well. + Increase limit for concurrently active TCP workers from 20 to 60. We've seen reports where 20 wasn't sufficient in user networks. Given that TCP workers do not really consume all that much more memory, this limit may even be increased further in case we recognize that 60 still isn't sufficient. + Use atomic_flag_test_and_set to ensure that FTL_TCP_worker_terminating() cannot run two times even when called exactly at the same time. + Wrap gravity reopening in locking to avoid a race collision with API requests (performed from independent threads). + Do not loop over known domains in addition to looping over all FTL-DNS-cache entries. + Factor out FTL sources into a separate object. This slightly enhances compilation speed. + Allow IPv6 addresses ofthe form [::ffff:1.2.3.4] in --dhcp-option. + Workaround for reported recvmsg() ignoring MSG_PEEK. + Log listening on new interfaces + Explicitly mark address port not used + Compare address and interface index for allowed interface + Cleanup interfaces no longer available + Handle listening on duplicate addresses + Remove duplicate address family from listener + Suppress logging of listen addresses during startup. + Apply floor of 60s to TTL of DNSKEY and DS records in cache. + Change default lease time for DHCPv6 to one day. + Do not try to bind a value when we should actually read one. This glitch was harmless, however, it prevented regex ID from being loaded from the database. + Create and use a temporary copy of the domain string during the analysis ------------------------------------------------------------------- Sun Jan 19 09:12:25 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Remove comments from pihole-FTL.conf ------------------------------------------------------------------- Sat Jan 18 09:42:22 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Apply ignore-shmem.c-strncpy-error.patch only for Tumbleweed ------------------------------------------------------------------- Sat Jan 18 09:09:37 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Added patch fix-build-after-y2038-changes-in-glib.patch * Modified from upstream to adjust to filename changes. - Added patch ignore-shmem.c-strncpy-error.patch * The code adds null byte, so that is not a problem - Added patch fix-build-with-libnettle-3.5.patch * Modified from upstream to adjust to filename changes. ------------------------------------------------------------------- Sat Jan 18 08:26:08 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Comment cleanup - Ignore version.h caused macro-in-comment - Change to arch specific package ------------------------------------------------------------------- Mon Jan 13 11:40:12 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Changed pid path to point to real pid in service file ------------------------------------------------------------------- Mon Jan 13 09:12:52 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Added noreplace to config clauses ------------------------------------------------------------------- Mon Jan 13 08:06:43 UTC 2020 - Samu Voutilainen <smar@smar.fi> - AdminLTE reads pihole-FTL.conf using PHP’s ini syntax ------------------------------------------------------------------- Fri Jan 10 11:58:06 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Added SUSE.readme for real ------------------------------------------------------------------- Fri Jan 10 09:01:53 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Added creation of /run/pihole via tmpfiles ------------------------------------------------------------------- Fri Jan 10 08:51:03 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Added undocumented file paths to supplied pihole-FTL.conf ------------------------------------------------------------------- Wed Jan 8 14:50:49 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Added commented /etc/permissions.d/pihole-ftl to instruct user about easy way of setting the permissions. ------------------------------------------------------------------- Wed Jan 8 13:30:11 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Added default pihole-FTL.conf ------------------------------------------------------------------- Wed Jan 8 12:03:01 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Patch Makefile to ignore the errors - Patch Makefile to contain dynamic libs ------------------------------------------------------------------- Wed Jan 8 11:56:47 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Generate version.h on the fly ------------------------------------------------------------------- Wed Jan 8 09:31:34 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Correct libnettle dependency ------------------------------------------------------------------- Wed Jan 8 09:31:20 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Removed comments ------------------------------------------------------------------- Wed Jan 8 09:07:26 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Added systemd service file ------------------------------------------------------------------- Wed Jan 8 08:43:00 UTC 2020 - Samu Voutilainen <smar@smar.fi> - First version to build ------------------------------------------------------------------- Wed Jan 8 07:36:17 UTC 2020 - Samu Voutilainen <smar@smar.fi> - Initial version
Locations
Projects
Search
Status Monitor
Help
OpenBuildService.org
Documentation
API Documentation
Contact
Support
@OBShq
The Open Build Service is an
openSUSE project
.
Sign Up
Log In
Places
Places
All Projects
Status Monitor
