RPCS3/llvm-mirror
1
0
Fork 0
mirror of https://github.com/RPCS3/llvm-mirror.git synced 2024-11-26 04:32:44 +01:00
Code Issues Projects Releases Wiki Activity
9b52b6c39a
llvm-mirror/unittests/Support/JSONTest.cpp

550 lines
17 KiB
C++
Raw Normal View History

Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
//===-- JSONTest.cpp - JSON unit tests --------------------------*- C++ -*-===//
//
Update the file headers across all of the LLVM projects in the monorepo to reflect the new license. We understand that people may be surprised that we're moving the header entirely to discuss the new license. We checked this carefully with the Foundation's lawyer and we believe this is the correct approach. Essentially, all code in the project is now made available by the LLVM project under our new license, so you will see that the license headers include that license only. Some of our contributors have contributed code under our old license, and accordingly, we have retained a copy of our old license notice in the top-level files in each project and repository. llvm-svn: 351636
2019-01-19 09:50:56 +01:00
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
//
//===----------------------------------------------------------------------===//
#include "llvm/Support/JSON.h"
[Support] Add JSON streaming output API, faster where the heavy value types aren't needed. Summary: There's still a little bit of constant factor that could be trimmed (e.g. more overloads to avoid round-tripping primitives through json::Value). But this solves the memory scaling problem, and greatly improves the performance constant factor, and the API should leave room for optimization if needed. Adapt TimeProfiler to use it, eliminating almost all the performance regression from r358476. Performance test on my machine: perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms json::Value (HEAD): 2757ms (+11%) After this patch: 2520 ms (+1.6%) Reviewers: anton-afanasyev, lebedev.ri Subscribers: kristina, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60804 llvm-svn: 359186
2019-04-25 14:51:42 +02:00
#include "llvm/Support/raw_ostream.h"
[JSON] Facility to track position within an object and report errors. This error model should be rich enough for most applications. It comprises: - a name for the root object, so the user knows what we're parsing - a path from the root object to the JSON node most associated with the error - a local error message This can be presented as an llvm::Error e.g. "expected string at ConfigFile.credentials[0].username" It's designed to be cheap: Paths are a linked list of lightweight objects on the stack. No heap allocations unless errors are encountered. A subsequent commit will make use of this in the JSON-to-object translation facilities: fromJSON and ObjectMapper. However it's independent of these and can be used for e.g. validation alone. Another subsequent commit will support showing the error in its context within the parsed value. Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 00:01:45 +02:00
#include "llvm/Testing/Support/Error.h"
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
#include "gmock/gmock.h"
#include "gtest/gtest.h"
namespace llvm {
namespace json {
namespace {
std::string s(const Value &E) { return llvm::formatv("{0}", E).str(); }
std::string sp(const Value &E) { return llvm::formatv("{0:2}", E).str(); }
TEST(JSONTest, Types) {
EXPECT_EQ("true", s(true));
EXPECT_EQ("null", s(nullptr));
EXPECT_EQ("2.5", s(2.5));
EXPECT_EQ(R"("foo")", s("foo"));
EXPECT_EQ("[1,2,3]", s({1, 2, 3}));
EXPECT_EQ(R"({"x":10,"y":20})", s(Object{{"x", 10}, {"y", 20}}));
[Support] Harded JSON against invalid UTF-8. Parsing invalid UTF-8 input is now a parse error. Creating JSON values from invalid UTF-8 now triggers an assertion, and (in no-assert builds) substitutes the unicode replacement character. Strings retrieved from json::Value are always valid UTF-8. llvm-svn: 336657
2018-07-10 13:51:26 +02:00
#ifdef NDEBUG
EXPECT_EQ(R"("<EFBFBD><EFBFBD>")", s("\xC0\x80"));
EXPECT_EQ(R"({"<EFBFBD><EFBFBD>":0})", s(Object{{"\xC0\x80", 0}}));
#else
EXPECT_DEATH(s("\xC0\x80"), "Invalid UTF-8");
EXPECT_DEATH(s(Object{{"\xC0\x80", 0}}), "Invalid UTF-8");
#endif
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
}
TEST(JSONTest, Constructors) {
// Lots of edge cases around empty and singleton init lists.
EXPECT_EQ("[[[3]]]", s({{{3}}}));
EXPECT_EQ("[[[]]]", s({{{}}}));
EXPECT_EQ("[[{}]]", s({{Object{}}}));
EXPECT_EQ(R"({"A":{"B":{}}})", s(Object{{"A", Object{{"B", Object{}}}}}));
EXPECT_EQ(R"({"A":{"B":{"X":"Y"}}})",
s(Object{{"A", Object{{"B", Object{{"X", "Y"}}}}}}));
[Support] Allow JSON serialization of Optional<T> for supported T. This is ported from r333881 to JSON's new home. llvm-svn: 336542
2018-07-09 14:26:09 +02:00
EXPECT_EQ("null", s(llvm::Optional<double>()));
EXPECT_EQ("2.5", s(llvm::Optional<double>(2.5)));
[Support] json::Value construction from std::vector<T> and std::map<string,T>. Summary: Previously this required a conversion to json::Array/json::Object first. Reviewers: ioeric Subscribers: kristina, llvm-commits Differential Revision: https://reviews.llvm.org/D53385 llvm-svn: 344732
2018-10-18 10:47:24 +02:00
EXPECT_EQ("[[2.5,null]]", s(std::vector<std::vector<llvm::Optional<double>>>{
{2.5, llvm::None}}));
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
}
TEST(JSONTest, StringOwnership) {
char X[] = "Hello";
Value Alias = static_cast<const char *>(X);
X[1] = 'a';
EXPECT_EQ(R"("Hallo")", s(Alias));
std::string Y = "Hello";
Value Copy = Y;
Y[1] = 'a';
EXPECT_EQ(R"("Hello")", s(Copy));
}
TEST(JSONTest, CanonicalOutput) {
// Objects are sorted (but arrays aren't)!
EXPECT_EQ(R"({"a":1,"b":2,"c":3})", s(Object{{"a", 1}, {"c", 3}, {"b", 2}}));
EXPECT_EQ(R"(["a","c","b"])", s({"a", "c", "b"}));
EXPECT_EQ("3", s(3.0));
}
TEST(JSONTest, Escaping) {
[Support] Add erase() to json::Object
2019-11-11 18:25:01 +01:00
std::string Test = {
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
0, // Strings may contain nulls.
'\b', '\f', // Have mnemonics, but we escape numerically.
'\r', '\n', '\t', // Escaped with mnemonics.
'S', '\"', '\\', // Printable ASCII characters.
'\x7f', // Delete is not escaped.
'\xce', '\x94', // Non-ASCII UTF-8 is not escaped.
};
[Support] Add erase() to json::Object
2019-11-11 18:25:01 +01:00
std::string TestString = R"("\u0000\u0008\u000c\r\n\tS\"\\)"
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
"\x7f\xCE\x94\"";
[Support] Add erase() to json::Object
2019-11-11 18:25:01 +01:00
EXPECT_EQ(TestString, s(Test));
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
EXPECT_EQ(R"({"object keys are\nescaped":true})",
s(Object{{"object keys are\nescaped", true}}));
}
TEST(JSONTest, PrettyPrinting) {
[Support] Add erase() to json::Object
2019-11-11 18:25:01 +01:00
const char Str[] = R"({
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
"empty_array": [],
"empty_object": {},
"full_array": [
1,
null
],
"full_object": {
"nested_array": [
{
"property": "value"
}
]
}
})";
[Support] Add erase() to json::Object
2019-11-11 18:25:01 +01:00
EXPECT_EQ(Str, sp(Object{
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
{"empty_object", Object{}},
{"empty_array", {}},
{"full_array", {1, nullptr}},
{"full_object",
Object{
{"nested_array",
{Object{
{"property", "value"},
}}},
}},
}));
}
[Support] Add erase() to json::Object
2019-11-11 18:25:01 +01:00
TEST(JSONTest, Array) {
Array A{1, 2};
A.emplace_back(3);
A.emplace(++A.begin(), 0);
A.push_back(4);
A.insert(++++A.begin(), 99);
EXPECT_EQ(A.size(), 6u);
EXPECT_EQ(R"([1,0,99,2,3,4])", s(std::move(A)));
}
TEST(JSONTest, Object) {
Object O{{"a", 1}, {"b", 2}, {"c", 3}};
EXPECT_TRUE(O.try_emplace("d", 4).second);
EXPECT_FALSE(O.try_emplace("a", 4).second);
auto D = O.find("d");
EXPECT_FALSE(D == O.end());
auto E = O.find("e");
EXPECT_TRUE(E == O.end());
O.erase("b");
O.erase(D);
EXPECT_EQ(O.size(), 2u);
EXPECT_EQ(R"({"a":1,"c":3})", s(std::move(O)));
}
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
TEST(JSONTest, Parse) {
auto Compare = [](llvm::StringRef S, Value Expected) {
if (auto E = parse(S)) {
// Compare both string forms and with operator==, in case we have bugs.
EXPECT_EQ(*E, Expected);
EXPECT_EQ(sp(*E), sp(Expected));
} else {
handleAllErrors(E.takeError(), [S](const llvm::ErrorInfoBase &E) {
FAIL() << "Failed to parse JSON >>> " << S << " <<<: " << E.message();
});
}
};
Compare(R"(true)", true);
Compare(R"(false)", false);
Compare(R"(null)", nullptr);
Compare(R"(42)", 42);
Compare(R"(2.5)", 2.5);
Compare(R"(2e50)", 2e50);
Compare(R"(1.2e3456789)", std::numeric_limits<double>::infinity());
Compare(R"("foo")", "foo");
Compare(R"("\"\\\b\f\n\r\t")", "\"\\\b\f\n\r\t");
Compare(R"("\u0000")", llvm::StringRef("\0", 1));
Compare("\"\x7f\"", "\x7f");
Compare(R"("\ud801\udc37")", u8"\U00010437"); // UTF16 surrogate pair escape.
Compare("\"\xE2\x82\xAC\xF0\x9D\x84\x9E\"", u8"\u20ac\U0001d11e"); // UTF8
Compare(
R"("LoneLeading=\ud801, LoneTrailing=\udc01, LeadingLeadingTrailing=\ud801\ud801\udc37")",
u8"LoneLeading=\ufffd, LoneTrailing=\ufffd, "
u8"LeadingLeadingTrailing=\ufffd\U00010437"); // Invalid unicode.
Compare(R"({"":0,"":0})", Object{{"", 0}});
Compare(R"({"obj":{},"arr":[]})", Object{{"obj", Object{}}, {"arr", {}}});
Compare(R"({"\n":{"\u0000":[[[[]]]]}})",
Object{{"\n", Object{
{llvm::StringRef("\0", 1), {{{{}}}}},
}}});
Compare("\r[\n\t] ", {});
}
TEST(JSONTest, ParseErrors) {
auto ExpectErr = [](llvm::StringRef Msg, llvm::StringRef S) {
if (auto E = parse(S)) {
// Compare both string forms and with operator==, in case we have bugs.
FAIL() << "Parsed JSON >>> " << S << " <<< but wanted error: " << Msg;
} else {
handleAllErrors(E.takeError(), [S, Msg](const llvm::ErrorInfoBase &E) {
Make llvm::StringRef to std::string conversions explicit. This is how it should've been and brings it more in line with std::string_view. There should be no functional change here. This is mostly mechanical from a custom clang-tidy check, with a lot of manual fixups. It uncovers a lot of minor inefficiencies. This doesn't actually modify StringRef yet, I'll do that in a follow-up.
2020-01-28 20:23:46 +01:00
EXPECT_THAT(E.message(), testing::HasSubstr(std::string(Msg))) << S;
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
});
}
};
ExpectErr("Unexpected EOF", "");
ExpectErr("Unexpected EOF", "[");
ExpectErr("Text after end of document", "[][]");
ExpectErr("Invalid JSON value (false?)", "fuzzy");
ExpectErr("Expected , or ]", "[2?]");
ExpectErr("Expected object key", "{a:2}");
ExpectErr("Expected : after object key", R"({"a",2})");
ExpectErr("Expected , or } after object property", R"({"a":2 "b":3})");
ExpectErr("Invalid JSON value", R"([&%!])");
ExpectErr("Invalid JSON value (number?)", "1e1.0");
ExpectErr("Unterminated string", R"("abc\"def)");
ExpectErr("Control character in string", "\"abc\ndef\"");
ExpectErr("Invalid escape sequence", R"("\030")");
ExpectErr("Invalid \\u escape sequence", R"("\usuck")");
ExpectErr("[3:3, byte=19]", R"({
"valid": 1,
invalid: 2
})");
[Support] Harded JSON against invalid UTF-8. Parsing invalid UTF-8 input is now a parse error. Creating JSON values from invalid UTF-8 now triggers an assertion, and (in no-assert builds) substitutes the unicode replacement character. Strings retrieved from json::Value are always valid UTF-8. llvm-svn: 336657
2018-07-10 13:51:26 +02:00
ExpectErr("Invalid UTF-8 sequence", "\"\xC0\x80\""); // WTF-8 null
}
// Direct tests of isUTF8 and fixUTF8. Internal uses are also tested elsewhere.
TEST(JSONTest, UTF8) {
for (const char *Valid : {
"this is ASCII text",
"thïs tëxt häs BMP chäräctërs",
"𐌶𐌰L𐌾𐍈 C𐍈𐌼𐌴𐍃",
}) {
EXPECT_TRUE(isUTF8(Valid)) << Valid;
EXPECT_EQ(fixUTF8(Valid), Valid);
}
for (auto Invalid : std::vector<std::pair<const char *, const char *>>{
{"lone trailing \x81\x82 bytes", "lone trailing <20><> bytes"},
{"missing trailing \xD0 bytes", "missing trailing <20> bytes"},
{"truncated character \xD0", "truncated character <20>"},
{"not \xC1\x80 the \xE0\x9f\xBF shortest \xF0\x83\x83\x83 encoding",
"not <20><> the <20><><EFBFBD> shortest <20><><EFBFBD><EFBFBD> encoding"},
{"too \xF9\x80\x80\x80\x80 long", "too <20><><EFBFBD><EFBFBD><EFBFBD> long"},
{"surrogate \xED\xA0\x80 invalid \xF4\x90\x80\x80",
"surrogate <20><><EFBFBD> invalid <20><><EFBFBD><EFBFBD>"}}) {
EXPECT_FALSE(isUTF8(Invalid.first)) << Invalid.first;
EXPECT_EQ(fixUTF8(Invalid.first), Invalid.second);
}
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
}
TEST(JSONTest, Inspection) {
llvm::Expected<Value> Doc = parse(R"(
{
"null": null,
"boolean": false,
"number": 2.78,
"string": "json",
"array": [null, true, 3.14, "hello", [1,2,3], {"time": "arrow"}],
"object": {"fruit": "banana"}
}
)");
EXPECT_TRUE(!!Doc);
Object *O = Doc->getAsObject();
ASSERT_TRUE(O);
EXPECT_FALSE(O->getNull("missing"));
EXPECT_FALSE(O->getNull("boolean"));
EXPECT_TRUE(O->getNull("null"));
EXPECT_EQ(O->getNumber("number"), llvm::Optional<double>(2.78));
EXPECT_FALSE(O->getInteger("number"));
EXPECT_EQ(O->getString("string"), llvm::Optional<llvm::StringRef>("json"));
ASSERT_FALSE(O->getObject("missing"));
ASSERT_FALSE(O->getObject("array"));
ASSERT_TRUE(O->getObject("object"));
EXPECT_EQ(*O->getObject("object"), (Object{{"fruit", "banana"}}));
Array *A = O->getArray("array");
ASSERT_TRUE(A);
EXPECT_EQ((*A)[1].getAsBoolean(), llvm::Optional<bool>(true));
ASSERT_TRUE((*A)[4].getAsArray());
EXPECT_EQ(*(*A)[4].getAsArray(), (Array{1, 2, 3}));
EXPECT_EQ((*(*A)[4].getAsArray())[1].getAsInteger(),
llvm::Optional<int64_t>(2));
int I = 0;
for (Value &E : *A) {
if (I++ == 5) {
ASSERT_TRUE(E.getAsObject());
EXPECT_EQ(E.getAsObject()->getString("time"),
llvm::Optional<llvm::StringRef>("arrow"));
} else
EXPECT_FALSE(E.getAsObject());
}
}
[Support] Make JSON handle doubles and int64s losslessly Summary: This patch adds a new "integer" ValueType, and renames Number -> Double. This allows us to preserve the full precision of int64_t when parsing integers from the wire, or constructing from an integer. The API is unchanged, other than giving asInteger() a clearer contract. In addition, always output doubles with enough precision that parsing will reconstruct the same double. Reviewers: simon_tatham Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D46209 llvm-svn: 336541
2018-07-09 14:16:40 +02:00
// Verify special integer handling - we try to preserve exact int64 values.
TEST(JSONTest, Integers) {
struct {
const char *Desc;
Value Val;
const char *Str;
llvm::Optional<int64_t> AsInt;
llvm::Optional<double> AsNumber;
} TestCases[] = {
{
"Non-integer. Stored as double, not convertible.",
double{1.5},
"1.5",
llvm::None,
1.5,
},
{
"Integer, not exact double. Stored as int64, convertible.",
int64_t{0x4000000000000001},
"4611686018427387905",
int64_t{0x4000000000000001},
double{0x4000000000000000},
},
{
"Negative integer, not exact double. Stored as int64, convertible.",
int64_t{-0x4000000000000001},
"-4611686018427387905",
int64_t{-0x4000000000000001},
double{-0x4000000000000000},
},
Disable a JSONTest.Integers test with newer MSVCs, PR46470
2020-06-27 01:57:52 +02:00
// PR46470,
// https://developercommunity.visualstudio.com/content/problem/1093399/incorrect-result-when-printing-6917529027641081856.html
Revert "Disable a JSONTest.Integers test with all MSVCs, PR46470" This reverts commit 63bcf89125fdfe8a41b0125454b006b49abb0184. Turns out the more targeted disablement in the previous change was good enough.
2020-06-27 04:30:24 +02:00
#if !defined(_MSC_VER) || _MSC_VER < 1926
[Support] Make JSON handle doubles and int64s losslessly Summary: This patch adds a new "integer" ValueType, and renames Number -> Double. This allows us to preserve the full precision of int64_t when parsing integers from the wire, or constructing from an integer. The API is unchanged, other than giving asInteger() a clearer contract. In addition, always output doubles with enough precision that parsing will reconstruct the same double. Reviewers: simon_tatham Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D46209 llvm-svn: 336541
2018-07-09 14:16:40 +02:00
{
"Dynamically exact integer. Stored as double, convertible.",
double{0x6000000000000000},
"6.9175290276410819e+18",
int64_t{0x6000000000000000},
double{0x6000000000000000},
},
Disable a JSONTest.Integers test with newer MSVCs, PR46470
2020-06-27 01:57:52 +02:00
#endif
[Support] Make JSON handle doubles and int64s losslessly Summary: This patch adds a new "integer" ValueType, and renames Number -> Double. This allows us to preserve the full precision of int64_t when parsing integers from the wire, or constructing from an integer. The API is unchanged, other than giving asInteger() a clearer contract. In addition, always output doubles with enough precision that parsing will reconstruct the same double. Reviewers: simon_tatham Subscribers: llvm-commits Differential Revision: https://reviews.llvm.org/D46209 llvm-svn: 336541
2018-07-09 14:16:40 +02:00
{
"Dynamically integer, >64 bits. Stored as double, not convertible.",
1.5 * double{0x8000000000000000},
"1.3835058055282164e+19",
llvm::None,
1.5 * double{0x8000000000000000},
},
};
for (const auto &T : TestCases) {
EXPECT_EQ(T.Str, s(T.Val)) << T.Desc;
llvm::Expected<Value> Doc = parse(T.Str);
EXPECT_TRUE(!!Doc) << T.Desc;
EXPECT_EQ(Doc->getAsInteger(), T.AsInt) << T.Desc;
EXPECT_EQ(Doc->getAsNumber(), T.AsNumber) << T.Desc;
EXPECT_EQ(T.Val, *Doc) << T.Desc;
EXPECT_EQ(T.Str, s(*Doc)) << T.Desc;
}
}
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
// Sample struct with typical JSON-mapping rules.
struct CustomStruct {
CustomStruct() : B(false) {}
CustomStruct(std::string S, llvm::Optional<int> I, bool B)
: S(S), I(I), B(B) {}
std::string S;
llvm::Optional<int> I;
bool B;
};
inline bool operator==(const CustomStruct &L, const CustomStruct &R) {
return L.S == R.S && L.I == R.I && L.B == R.B;
}
inline llvm::raw_ostream &operator<<(llvm::raw_ostream &OS,
const CustomStruct &S) {
return OS << "(" << S.S << ", " << (S.I ? std::to_string(*S.I) : "None")
<< ", " << S.B << ")";
}
[JSON] Add error reporting to fromJSON and ObjectMapper Translating between JSON objects and C++ strutctures is common. From experience in clangd, fromJSON/ObjectMapper work well and save a lot of code, but aren't adopted elsewhere at least partly due to total lack of error reporting beyond "ok"/"bad". The recently-added error model should be rich enough for most applications. It requires tracking the path within the root object and reporting local errors at appropriate places. To do this, we exploit the fact that the call graph of recursive parse functions mirror the structure of the JSON itself. The current path is represented as a linked list of segments, each of which is on the stack as a parameter. Concretely, fromJSON now looks like: bool fromJSON(const Value&, T&, Path); Beyond the signature change, this is reasonably unobtrusive: building the path segments is mostly handled by ObjectMapper and the vector<T> fromJSON. However the root caller of fromJSON must now create a Root object to store the errors, which is a little clunky. I've added high-level parse<T>(StringRef) -> Expected<T>, but it's not general enough to be the primary interface I think (at least, not usable in clangd). All existing users (mostly just clangd) are updated in this patch, making this change backwards-compatible is a bit hairy. Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 01:14:12 +02:00
bool fromJSON(const Value &E, CustomStruct &R, Path P) {
ObjectMapper O(E, P);
[JSON] Add ObjectMapper::mapOptional to validate optional data. Currently the idiom for mapping optional fields is: ObjectMapper O(Val, P); if (!O.map("required1", Out.R1) || !O.map("required2", Out.R2)) return false; O.map("optional1", Out.O1); // ignore result return true; If `optional1` is present but malformed, then we won't detect/report that error. We may even leave `Out` in an incomplete state while returning true. Instead, we'd often prefer to ignore `optional1` if it is absent, but otherwise behave just like map(). Differential Revision: https://reviews.llvm.org/D89128
2020-10-09 15:33:56 +02:00
return O && O.map("str", R.S) && O.map("int", R.I) &&
O.mapOptional("bool", R.B);
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
}
[JSON] Add error reporting to fromJSON and ObjectMapper Translating between JSON objects and C++ strutctures is common. From experience in clangd, fromJSON/ObjectMapper work well and save a lot of code, but aren't adopted elsewhere at least partly due to total lack of error reporting beyond "ok"/"bad". The recently-added error model should be rich enough for most applications. It requires tracking the path within the root object and reporting local errors at appropriate places. To do this, we exploit the fact that the call graph of recursive parse functions mirror the structure of the JSON itself. The current path is represented as a linked list of segments, each of which is on the stack as a parameter. Concretely, fromJSON now looks like: bool fromJSON(const Value&, T&, Path); Beyond the signature change, this is reasonably unobtrusive: building the path segments is mostly handled by ObjectMapper and the vector<T> fromJSON. However the root caller of fromJSON must now create a Root object to store the errors, which is a little clunky. I've added high-level parse<T>(StringRef) -> Expected<T>, but it's not general enough to be the primary interface I think (at least, not usable in clangd). All existing users (mostly just clangd) are updated in this patch, making this change backwards-compatible is a bit hairy. Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 01:14:12 +02:00
static std::string errorContext(const Value &V, const Path::Root &R) {
std::string Context;
llvm::raw_string_ostream OS(Context);
R.printErrorContext(V, OS);
return OS.str();
}
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
TEST(JSONTest, Deserialize) {
std::map<std::string, std::vector<CustomStruct>> R;
CustomStruct ExpectedStruct = {"foo", 42, true};
std::map<std::string, std::vector<CustomStruct>> Expected;
[JSON] Add ObjectMapper::mapOptional to validate optional data. Currently the idiom for mapping optional fields is: ObjectMapper O(Val, P); if (!O.map("required1", Out.R1) || !O.map("required2", Out.R2)) return false; O.map("optional1", Out.O1); // ignore result return true; If `optional1` is present but malformed, then we won't detect/report that error. We may even leave `Out` in an incomplete state while returning true. Instead, we'd often prefer to ignore `optional1` if it is absent, but otherwise behave just like map(). Differential Revision: https://reviews.llvm.org/D89128
2020-10-09 15:33:56 +02:00
Value J = Object{{"foo", Array{
Object{
{"str", "foo"},
{"int", 42},
{"bool", true},
{"unknown", "ignored"},
},
Object{{"str", "bar"}},
}}};
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
Expected["foo"] = {
CustomStruct("foo", 42, true),
CustomStruct("bar", llvm::None, false),
};
[JSON] Add error reporting to fromJSON and ObjectMapper Translating between JSON objects and C++ strutctures is common. From experience in clangd, fromJSON/ObjectMapper work well and save a lot of code, but aren't adopted elsewhere at least partly due to total lack of error reporting beyond "ok"/"bad". The recently-added error model should be rich enough for most applications. It requires tracking the path within the root object and reporting local errors at appropriate places. To do this, we exploit the fact that the call graph of recursive parse functions mirror the structure of the JSON itself. The current path is represented as a linked list of segments, each of which is on the stack as a parameter. Concretely, fromJSON now looks like: bool fromJSON(const Value&, T&, Path); Beyond the signature change, this is reasonably unobtrusive: building the path segments is mostly handled by ObjectMapper and the vector<T> fromJSON. However the root caller of fromJSON must now create a Root object to store the errors, which is a little clunky. I've added high-level parse<T>(StringRef) -> Expected<T>, but it's not general enough to be the primary interface I think (at least, not usable in clangd). All existing users (mostly just clangd) are updated in this patch, making this change backwards-compatible is a bit hairy. Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 01:14:12 +02:00
Path::Root Root("CustomStruct");
ASSERT_TRUE(fromJSON(J, R, Root));
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
EXPECT_EQ(R, Expected);
[JSON] Add error reporting to fromJSON and ObjectMapper Translating between JSON objects and C++ strutctures is common. From experience in clangd, fromJSON/ObjectMapper work well and save a lot of code, but aren't adopted elsewhere at least partly due to total lack of error reporting beyond "ok"/"bad". The recently-added error model should be rich enough for most applications. It requires tracking the path within the root object and reporting local errors at appropriate places. To do this, we exploit the fact that the call graph of recursive parse functions mirror the structure of the JSON itself. The current path is represented as a linked list of segments, each of which is on the stack as a parameter. Concretely, fromJSON now looks like: bool fromJSON(const Value&, T&, Path); Beyond the signature change, this is reasonably unobtrusive: building the path segments is mostly handled by ObjectMapper and the vector<T> fromJSON. However the root caller of fromJSON must now create a Root object to store the errors, which is a little clunky. I've added high-level parse<T>(StringRef) -> Expected<T>, but it's not general enough to be the primary interface I think (at least, not usable in clangd). All existing users (mostly just clangd) are updated in this patch, making this change backwards-compatible is a bit hairy. Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 01:14:12 +02:00
(*J.getAsObject()->getArray("foo"))[0] = 123;
ASSERT_FALSE(fromJSON(J, R, Root));
EXPECT_EQ("expected object at CustomStruct.foo[0]",
toString(Root.getError()));
const char *ExpectedDump = R"({
"foo": [
/* error: expected object */
123,
{ ... }
]
})";
EXPECT_EQ(ExpectedDump, errorContext(J, Root));
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
CustomStruct V;
[JSON] Add error reporting to fromJSON and ObjectMapper Translating between JSON objects and C++ strutctures is common. From experience in clangd, fromJSON/ObjectMapper work well and save a lot of code, but aren't adopted elsewhere at least partly due to total lack of error reporting beyond "ok"/"bad". The recently-added error model should be rich enough for most applications. It requires tracking the path within the root object and reporting local errors at appropriate places. To do this, we exploit the fact that the call graph of recursive parse functions mirror the structure of the JSON itself. The current path is represented as a linked list of segments, each of which is on the stack as a parameter. Concretely, fromJSON now looks like: bool fromJSON(const Value&, T&, Path); Beyond the signature change, this is reasonably unobtrusive: building the path segments is mostly handled by ObjectMapper and the vector<T> fromJSON. However the root caller of fromJSON must now create a Root object to store the errors, which is a little clunky. I've added high-level parse<T>(StringRef) -> Expected<T>, but it's not general enough to be the primary interface I think (at least, not usable in clangd). All existing users (mostly just clangd) are updated in this patch, making this change backwards-compatible is a bit hairy. Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 01:14:12 +02:00
EXPECT_FALSE(fromJSON(nullptr, V, Root));
EXPECT_EQ("expected object when parsing CustomStruct",
toString(Root.getError()));
EXPECT_FALSE(fromJSON(Object{}, V, Root));
EXPECT_EQ("missing value at CustomStruct.str", toString(Root.getError()));
EXPECT_FALSE(fromJSON(Object{{"str", 1}}, V, Root));
EXPECT_EQ("expected string at CustomStruct.str", toString(Root.getError()));
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
// Optional<T> must parse as the correct type if present.
[JSON] Add error reporting to fromJSON and ObjectMapper Translating between JSON objects and C++ strutctures is common. From experience in clangd, fromJSON/ObjectMapper work well and save a lot of code, but aren't adopted elsewhere at least partly due to total lack of error reporting beyond "ok"/"bad". The recently-added error model should be rich enough for most applications. It requires tracking the path within the root object and reporting local errors at appropriate places. To do this, we exploit the fact that the call graph of recursive parse functions mirror the structure of the JSON itself. The current path is represented as a linked list of segments, each of which is on the stack as a parameter. Concretely, fromJSON now looks like: bool fromJSON(const Value&, T&, Path); Beyond the signature change, this is reasonably unobtrusive: building the path segments is mostly handled by ObjectMapper and the vector<T> fromJSON. However the root caller of fromJSON must now create a Root object to store the errors, which is a little clunky. I've added high-level parse<T>(StringRef) -> Expected<T>, but it's not general enough to be the primary interface I think (at least, not usable in clangd). All existing users (mostly just clangd) are updated in this patch, making this change backwards-compatible is a bit hairy. Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 01:14:12 +02:00
EXPECT_FALSE(fromJSON(Object{{"str", "1"}, {"int", "string"}}, V, Root));
EXPECT_EQ("expected integer at CustomStruct.int", toString(Root.getError()));
[JSON] Add ObjectMapper::mapOptional to validate optional data. Currently the idiom for mapping optional fields is: ObjectMapper O(Val, P); if (!O.map("required1", Out.R1) || !O.map("required2", Out.R2)) return false; O.map("optional1", Out.O1); // ignore result return true; If `optional1` is present but malformed, then we won't detect/report that error. We may even leave `Out` in an incomplete state while returning true. Instead, we'd often prefer to ignore `optional1` if it is absent, but otherwise behave just like map(). Differential Revision: https://reviews.llvm.org/D89128
2020-10-09 15:33:56 +02:00
// mapOptional must parse as the correct type if present.
EXPECT_FALSE(fromJSON(Object{{"str", "1"}, {"bool", "string"}}, V, Root));
EXPECT_EQ("expected boolean at CustomStruct.bool", toString(Root.getError()));
[JSON] Add error reporting to fromJSON and ObjectMapper Translating between JSON objects and C++ strutctures is common. From experience in clangd, fromJSON/ObjectMapper work well and save a lot of code, but aren't adopted elsewhere at least partly due to total lack of error reporting beyond "ok"/"bad". The recently-added error model should be rich enough for most applications. It requires tracking the path within the root object and reporting local errors at appropriate places. To do this, we exploit the fact that the call graph of recursive parse functions mirror the structure of the JSON itself. The current path is represented as a linked list of segments, each of which is on the stack as a parameter. Concretely, fromJSON now looks like: bool fromJSON(const Value&, T&, Path); Beyond the signature change, this is reasonably unobtrusive: building the path segments is mostly handled by ObjectMapper and the vector<T> fromJSON. However the root caller of fromJSON must now create a Root object to store the errors, which is a little clunky. I've added high-level parse<T>(StringRef) -> Expected<T>, but it's not general enough to be the primary interface I think (at least, not usable in clangd). All existing users (mostly just clangd) are updated in this patch, making this change backwards-compatible is a bit hairy. Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 01:14:12 +02:00
}
TEST(JSONTest, ParseDeserialize) {
auto E = parse<std::vector<CustomStruct>>(R"json(
[{"str": "foo", "int": 42}, {"int": 42}]
)json");
EXPECT_THAT_EXPECTED(E, FailedWithMessage("missing value at (root)[1].str"));
E = parse<std::vector<CustomStruct>>(R"json(
[{"str": "foo", "int": 42}, {"str": "bar"}
)json");
EXPECT_THAT_EXPECTED(
E,
FailedWithMessage("[3:2, byte=50]: Expected , or ] after array element"));
E = parse<std::vector<CustomStruct>>(R"json(
[{"str": "foo", "int": 42}]
)json");
EXPECT_THAT_EXPECTED(E, Succeeded());
EXPECT_THAT(*E, testing::SizeIs(1));
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
}
[Support] Add JSON streaming output API, faster where the heavy value types aren't needed. Summary: There's still a little bit of constant factor that could be trimmed (e.g. more overloads to avoid round-tripping primitives through json::Value). But this solves the memory scaling problem, and greatly improves the performance constant factor, and the API should leave room for optimization if needed. Adapt TimeProfiler to use it, eliminating almost all the performance regression from r358476. Performance test on my machine: perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms json::Value (HEAD): 2757ms (+11%) After this patch: 2520 ms (+1.6%) Reviewers: anton-afanasyev, lebedev.ri Subscribers: kristina, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60804 llvm-svn: 359186
2019-04-25 14:51:42 +02:00
TEST(JSONTest, Stream) {
auto StreamStuff = [](unsigned Indent) {
std::string S;
llvm::raw_string_ostream OS(S);
OStream J(OS, Indent);
[JSON] Allow emitting comments in json::OStream This isn't standard JSON, but is a popular extension. It will be used to show errors in context, rendering pseudo-json for humans. Differential Revision: https://reviews.llvm.org/D88103
2020-09-23 23:33:19 +02:00
J.comment("top*/level");
[Support] Add JSON streaming output API, faster where the heavy value types aren't needed. Summary: There's still a little bit of constant factor that could be trimmed (e.g. more overloads to avoid round-tripping primitives through json::Value). But this solves the memory scaling problem, and greatly improves the performance constant factor, and the API should leave room for optimization if needed. Adapt TimeProfiler to use it, eliminating almost all the performance regression from r358476. Performance test on my machine: perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms json::Value (HEAD): 2757ms (+11%) After this patch: 2520 ms (+1.6%) Reviewers: anton-afanasyev, lebedev.ri Subscribers: kristina, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60804 llvm-svn: 359186
2019-04-25 14:51:42 +02:00
J.object([&] {
J.attributeArray("foo", [&] {
J.value(nullptr);
[JSON] Allow emitting comments in json::OStream This isn't standard JSON, but is a popular extension. It will be used to show errors in context, rendering pseudo-json for humans. Differential Revision: https://reviews.llvm.org/D88103
2020-09-23 23:33:19 +02:00
J.comment("element");
[Support] Add JSON streaming output API, faster where the heavy value types aren't needed. Summary: There's still a little bit of constant factor that could be trimmed (e.g. more overloads to avoid round-tripping primitives through json::Value). But this solves the memory scaling problem, and greatly improves the performance constant factor, and the API should leave room for optimization if needed. Adapt TimeProfiler to use it, eliminating almost all the performance regression from r358476. Performance test on my machine: perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms json::Value (HEAD): 2757ms (+11%) After this patch: 2520 ms (+1.6%) Reviewers: anton-afanasyev, lebedev.ri Subscribers: kristina, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60804 llvm-svn: 359186
2019-04-25 14:51:42 +02:00
J.value(42.5);
J.arrayBegin();
J.value(43);
J.arrayEnd();
[json] Provide a means to delegate writing a value to another API (Based on D87170 by dsanders) I recently had need to call out to an external API to emit a JSON object as part of one an LLVM tool was emitting. However, our JSON support didn't provide a way to delegate part of the JSON output to that API. Add rawValueBegin() and rawValueEnd() to maintain and check the internal state while something else is writing to the stream. It's the users responsibility to ensure that the resulting JSON output is still valid. Differential Revision: https://reviews.llvm.org/D88902
2020-10-06 13:50:41 +02:00
J.rawValue([](raw_ostream &OS) { OS << "'unverified\nraw value'"; });
[Support] Add JSON streaming output API, faster where the heavy value types aren't needed. Summary: There's still a little bit of constant factor that could be trimmed (e.g. more overloads to avoid round-tripping primitives through json::Value). But this solves the memory scaling problem, and greatly improves the performance constant factor, and the API should leave room for optimization if needed. Adapt TimeProfiler to use it, eliminating almost all the performance regression from r358476. Performance test on my machine: perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms json::Value (HEAD): 2757ms (+11%) After this patch: 2520 ms (+1.6%) Reviewers: anton-afanasyev, lebedev.ri Subscribers: kristina, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60804 llvm-svn: 359186
2019-04-25 14:51:42 +02:00
});
[JSON] Allow emitting comments in json::OStream This isn't standard JSON, but is a popular extension. It will be used to show errors in context, rendering pseudo-json for humans. Differential Revision: https://reviews.llvm.org/D88103
2020-09-23 23:33:19 +02:00
J.comment("attribute");
[Support] Add JSON streaming output API, faster where the heavy value types aren't needed. Summary: There's still a little bit of constant factor that could be trimmed (e.g. more overloads to avoid round-tripping primitives through json::Value). But this solves the memory scaling problem, and greatly improves the performance constant factor, and the API should leave room for optimization if needed. Adapt TimeProfiler to use it, eliminating almost all the performance regression from r358476. Performance test on my machine: perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms json::Value (HEAD): 2757ms (+11%) After this patch: 2520 ms (+1.6%) Reviewers: anton-afanasyev, lebedev.ri Subscribers: kristina, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60804 llvm-svn: 359186
2019-04-25 14:51:42 +02:00
J.attributeBegin("bar");
[JSON] Allow emitting comments in json::OStream This isn't standard JSON, but is a popular extension. It will be used to show errors in context, rendering pseudo-json for humans. Differential Revision: https://reviews.llvm.org/D88103
2020-09-23 23:33:19 +02:00
J.comment("attribute value");
[Support] Add JSON streaming output API, faster where the heavy value types aren't needed. Summary: There's still a little bit of constant factor that could be trimmed (e.g. more overloads to avoid round-tripping primitives through json::Value). But this solves the memory scaling problem, and greatly improves the performance constant factor, and the API should leave room for optimization if needed. Adapt TimeProfiler to use it, eliminating almost all the performance regression from r358476. Performance test on my machine: perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms json::Value (HEAD): 2757ms (+11%) After this patch: 2520 ms (+1.6%) Reviewers: anton-afanasyev, lebedev.ri Subscribers: kristina, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60804 llvm-svn: 359186
2019-04-25 14:51:42 +02:00
J.objectBegin();
J.objectEnd();
J.attributeEnd();
J.attribute("baz", "xyz");
});
return OS.str();
};
[JSON] Allow emitting comments in json::OStream This isn't standard JSON, but is a popular extension. It will be used to show errors in context, rendering pseudo-json for humans. Differential Revision: https://reviews.llvm.org/D88103
2020-09-23 23:33:19 +02:00
const char *Plain =
[json] Provide a means to delegate writing a value to another API (Based on D87170 by dsanders) I recently had need to call out to an external API to emit a JSON object as part of one an LLVM tool was emitting. However, our JSON support didn't provide a way to delegate part of the JSON output to that API. Add rawValueBegin() and rawValueEnd() to maintain and check the internal state while something else is writing to the stream. It's the users responsibility to ensure that the resulting JSON output is still valid. Differential Revision: https://reviews.llvm.org/D88902
2020-10-06 13:50:41 +02:00
R"(/*top* /level*/{"foo":[null,/*element*/42.5,[43],'unverified
raw value'],/*attribute*/"bar":/*attribute value*/{},"baz":"xyz"})";
[Support] Add JSON streaming output API, faster where the heavy value types aren't needed. Summary: There's still a little bit of constant factor that could be trimmed (e.g. more overloads to avoid round-tripping primitives through json::Value). But this solves the memory scaling problem, and greatly improves the performance constant factor, and the API should leave room for optimization if needed. Adapt TimeProfiler to use it, eliminating almost all the performance regression from r358476. Performance test on my machine: perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms json::Value (HEAD): 2757ms (+11%) After this patch: 2520 ms (+1.6%) Reviewers: anton-afanasyev, lebedev.ri Subscribers: kristina, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60804 llvm-svn: 359186
2019-04-25 14:51:42 +02:00
EXPECT_EQ(Plain, StreamStuff(0));
[JSON] Allow emitting comments in json::OStream This isn't standard JSON, but is a popular extension. It will be used to show errors in context, rendering pseudo-json for humans. Differential Revision: https://reviews.llvm.org/D88103
2020-09-23 23:33:19 +02:00
const char *Pretty = R"(/* top* /level */
{
[Support] Add JSON streaming output API, faster where the heavy value types aren't needed. Summary: There's still a little bit of constant factor that could be trimmed (e.g. more overloads to avoid round-tripping primitives through json::Value). But this solves the memory scaling problem, and greatly improves the performance constant factor, and the API should leave room for optimization if needed. Adapt TimeProfiler to use it, eliminating almost all the performance regression from r358476. Performance test on my machine: perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms json::Value (HEAD): 2757ms (+11%) After this patch: 2520 ms (+1.6%) Reviewers: anton-afanasyev, lebedev.ri Subscribers: kristina, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60804 llvm-svn: 359186
2019-04-25 14:51:42 +02:00
"foo": [
null,
[JSON] Allow emitting comments in json::OStream This isn't standard JSON, but is a popular extension. It will be used to show errors in context, rendering pseudo-json for humans. Differential Revision: https://reviews.llvm.org/D88103
2020-09-23 23:33:19 +02:00
/* element */
[Support] Add JSON streaming output API, faster where the heavy value types aren't needed. Summary: There's still a little bit of constant factor that could be trimmed (e.g. more overloads to avoid round-tripping primitives through json::Value). But this solves the memory scaling problem, and greatly improves the performance constant factor, and the API should leave room for optimization if needed. Adapt TimeProfiler to use it, eliminating almost all the performance regression from r358476. Performance test on my machine: perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms json::Value (HEAD): 2757ms (+11%) After this patch: 2520 ms (+1.6%) Reviewers: anton-afanasyev, lebedev.ri Subscribers: kristina, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60804 llvm-svn: 359186
2019-04-25 14:51:42 +02:00
42.5,
[
43
[json] Provide a means to delegate writing a value to another API (Based on D87170 by dsanders) I recently had need to call out to an external API to emit a JSON object as part of one an LLVM tool was emitting. However, our JSON support didn't provide a way to delegate part of the JSON output to that API. Add rawValueBegin() and rawValueEnd() to maintain and check the internal state while something else is writing to the stream. It's the users responsibility to ensure that the resulting JSON output is still valid. Differential Revision: https://reviews.llvm.org/D88902
2020-10-06 13:50:41 +02:00
],
'unverified
raw value'
[Support] Add JSON streaming output API, faster where the heavy value types aren't needed. Summary: There's still a little bit of constant factor that could be trimmed (e.g. more overloads to avoid round-tripping primitives through json::Value). But this solves the memory scaling problem, and greatly improves the performance constant factor, and the API should leave room for optimization if needed. Adapt TimeProfiler to use it, eliminating almost all the performance regression from r358476. Performance test on my machine: perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms json::Value (HEAD): 2757ms (+11%) After this patch: 2520 ms (+1.6%) Reviewers: anton-afanasyev, lebedev.ri Subscribers: kristina, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60804 llvm-svn: 359186
2019-04-25 14:51:42 +02:00
],
[JSON] Allow emitting comments in json::OStream This isn't standard JSON, but is a popular extension. It will be used to show errors in context, rendering pseudo-json for humans. Differential Revision: https://reviews.llvm.org/D88103
2020-09-23 23:33:19 +02:00
/* attribute */
"bar": /* attribute value */ {},
[Support] Add JSON streaming output API, faster where the heavy value types aren't needed. Summary: There's still a little bit of constant factor that could be trimmed (e.g. more overloads to avoid round-tripping primitives through json::Value). But this solves the memory scaling problem, and greatly improves the performance constant factor, and the API should leave room for optimization if needed. Adapt TimeProfiler to use it, eliminating almost all the performance regression from r358476. Performance test on my machine: perf stat -r 5 ~/llvmbuild-opt/bin/clang++ -w -S -ftime-trace -mllvm -time-trace-granularity=0 spirit.cpp Handcrafted JSON (HEAD=r358532 with r358476 reverted): 2480ms json::Value (HEAD): 2757ms (+11%) After this patch: 2520 ms (+1.6%) Reviewers: anton-afanasyev, lebedev.ri Subscribers: kristina, llvm-commits Tags: #llvm Differential Revision: https://reviews.llvm.org/D60804 llvm-svn: 359186
2019-04-25 14:51:42 +02:00
"baz": "xyz"
})";
EXPECT_EQ(Pretty, StreamStuff(2));
}
[JSON] Facility to track position within an object and report errors. This error model should be rich enough for most applications. It comprises: - a name for the root object, so the user knows what we're parsing - a path from the root object to the JSON node most associated with the error - a local error message This can be presented as an llvm::Error e.g. "expected string at ConfigFile.credentials[0].username" It's designed to be cheap: Paths are a linked list of lightweight objects on the stack. No heap allocations unless errors are encountered. A subsequent commit will make use of this in the JSON-to-object translation facilities: fromJSON and ObjectMapper. However it's independent of these and can be used for e.g. validation alone. Another subsequent commit will support showing the error in its context within the parsed value. Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 00:01:45 +02:00
TEST(JSONTest, Path) {
Path::Root R("foo");
Path P = R, A = P.field("a"), B = P.field("b");
[JSON] Display errors associated with Paths in context When an error occurs processing a JSON object, seeing the actual surrounding data helps. Dumping just the node where the problem was identified can be too much or too little information. printErrorContext() shows the error message in its context, as a comment. JSON values along the path to the broken place are shown in some detail, the rest of the document is elided. For example: ``` { "credentials": [ { "username": /* error: expected string */ 42, "password": "secret" }, { ... } ] "backups": { ... } } ``` Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 00:28:52 +02:00
P.report("oh no");
EXPECT_THAT_ERROR(R.getError(), FailedWithMessage("oh no when parsing foo"));
[JSON] Facility to track position within an object and report errors. This error model should be rich enough for most applications. It comprises: - a name for the root object, so the user knows what we're parsing - a path from the root object to the JSON node most associated with the error - a local error message This can be presented as an llvm::Error e.g. "expected string at ConfigFile.credentials[0].username" It's designed to be cheap: Paths are a linked list of lightweight objects on the stack. No heap allocations unless errors are encountered. A subsequent commit will make use of this in the JSON-to-object translation facilities: fromJSON and ObjectMapper. However it's independent of these and can be used for e.g. validation alone. Another subsequent commit will support showing the error in its context within the parsed value. Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 00:01:45 +02:00
A.index(1).field("c").index(2).report("boom");
EXPECT_THAT_ERROR(R.getError(), FailedWithMessage("boom at foo.a[1].c[2]"));
B.field("d").field("e").report("bam");
EXPECT_THAT_ERROR(R.getError(), FailedWithMessage("bam at foo.b.d.e"));
[JSON] Display errors associated with Paths in context When an error occurs processing a JSON object, seeing the actual surrounding data helps. Dumping just the node where the problem was identified can be too much or too little information. printErrorContext() shows the error message in its context, as a comment. JSON values along the path to the broken place are shown in some detail, the rest of the document is elided. For example: ``` { "credentials": [ { "username": /* error: expected string */ 42, "password": "secret" }, { ... } ] "backups": { ... } } ``` Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 00:28:52 +02:00
Value V = Object{
{"a", Array{42}},
{"b",
Object{{"d",
Object{
{"e", Array{1, Object{{"x", "y"}}}},
{"f", "a moderately long string: 48 characters in total"},
}}}},
};
const char *Expected = R"({
"a": [ ... ],
"b": {
"d": {
"e": /* error: bam */ [
1,
{ ... }
],
"f": "a moderately long string: 48 characte..."
}
}
})";
[JSON] Add error reporting to fromJSON and ObjectMapper Translating between JSON objects and C++ strutctures is common. From experience in clangd, fromJSON/ObjectMapper work well and save a lot of code, but aren't adopted elsewhere at least partly due to total lack of error reporting beyond "ok"/"bad". The recently-added error model should be rich enough for most applications. It requires tracking the path within the root object and reporting local errors at appropriate places. To do this, we exploit the fact that the call graph of recursive parse functions mirror the structure of the JSON itself. The current path is represented as a linked list of segments, each of which is on the stack as a parameter. Concretely, fromJSON now looks like: bool fromJSON(const Value&, T&, Path); Beyond the signature change, this is reasonably unobtrusive: building the path segments is mostly handled by ObjectMapper and the vector<T> fromJSON. However the root caller of fromJSON must now create a Root object to store the errors, which is a little clunky. I've added high-level parse<T>(StringRef) -> Expected<T>, but it's not general enough to be the primary interface I think (at least, not usable in clangd). All existing users (mostly just clangd) are updated in this patch, making this change backwards-compatible is a bit hairy. Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 01:14:12 +02:00
EXPECT_EQ(Expected, errorContext(V, R));
[JSON] Facility to track position within an object and report errors. This error model should be rich enough for most applications. It comprises: - a name for the root object, so the user knows what we're parsing - a path from the root object to the JSON node most associated with the error - a local error message This can be presented as an llvm::Error e.g. "expected string at ConfigFile.credentials[0].username" It's designed to be cheap: Paths are a linked list of lightweight objects on the stack. No heap allocations unless errors are encountered. A subsequent commit will make use of this in the JSON-to-object translation facilities: fromJSON and ObjectMapper. However it's independent of these and can be used for e.g. validation alone. Another subsequent commit will support showing the error in its context within the parsed value. Differential Revision: https://reviews.llvm.org/D88103
2020-09-24 00:01:45 +02:00
}
Lift JSON library from clang-tools-extra/clangd to llvm/Support. Summary: This consists of four main parts: - an type json::Expr representing JSON values of dynamic kind, which can be composed, inspected, and modified - a JSON parser from string -> json::Expr - a JSON printer from json::Expr -> string, with optional pretty-printing - a convention for mapping json::Expr <=> native types (fromJSON/toJSON) Mapping functions are provided for primitives (e.g. int, vector) and the ObjectMapper helper helps implement fromJSON for struct/object types. Based on clangd's usage, a couple of places I'd appreciate review attention: - fromJSON returns only bool. A richer error-signaling mechanism may be useful to provide useful messages, or let recursive fromJSONs (containers/structs) do careful error recovery. - should json::obj be always explicitly written (like json::ary) - there's no streaming parse API. I suspect there are some simple wins like a callback API where the document is a long array, and each element is small. But this can probably be bolted on easily when we see the need. Reviewers: bkramer, labath Subscribers: mgorny, ilya-biryukov, ioeric, MaskRay, llvm-commits Differential Revision: https://reviews.llvm.org/D45753 llvm-svn: 336534
2018-07-09 12:05:41 +02:00
} // namespace
} // namespace json
} // namespace llvm
Reference in New Issue Copy Permalink