fix: Default decode any UTF-8 characters

[medz]

Refs: #789

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[natebosch]

This is a breaking behavior change.

[natebosch]

@brianquinlan - this might be worth trying to land while revving to 1.0.0 - I think there were only a few internal breakages.

I had a different fix for this that I was trying, I think this PR would break usage of the text content type.

The fix I was running with was

diff --git a/pkgs/http/lib/src/response.dart b/pkgs/http/lib/src/response.dart
index 1ba7c46..48ecb03 100644
--- a/pkgs/http/lib/src/response.dart
+++ b/pkgs/http/lib/src/response.dart
@@ -68,8 +68,11 @@ class Response extends BaseResponse {
 ///
 /// Defaults to [latin1] if the headers don't specify a charset or if that
 /// charset is unknown.
-Encoding _encodingForHeaders(Map<String, String> headers) =>
-    encodingForCharset(_contentTypeForHeaders(headers).parameters['charset']);
+Encoding _encodingForHeaders(Map<String, String> headers) {
+  final contentType = _contentTypeForHeaders(headers);
+  final defaultEncoding = contentType.type == 'text' ? latin1 : utf8;
+  return encodingForCharset(contentType.parameters['charset'], defaultEncoding);
+}
 
 /// Returns the [MediaType] object for the given headers's content-type.
 ///
diff --git a/pkgs/http/lib/src/utils.dart b/pkgs/http/lib/src/utils.dart
index e79108e..0d8f748 100644
--- a/pkgs/http/lib/src/utils.dart
+++ b/pkgs/http/lib/src/utils.dart
@@ -25,7 +25,7 @@ String mapToQuery(Map<String, String> map, {Encoding? encoding}) {
 ///
 /// Returns [fallback] if [charset] is null or if no [Encoding] was found that
 /// corresponds to [charset].
-Encoding encodingForCharset(String? charset, [Encoding fallback = latin1]) {
+Encoding encodingForCharset(String? charset, Encoding fallback) {
   if (charset == null) return fallback;
   return Encoding.getByName(charset) ?? fallback;
 }

cl/385011986 is a (stale) CL if you'd like to try it out internally