Please don’t follow the advice given in the InfoQ article on building an Avro schema up from multiple files. The article recommends doing string replacement to mutate the schemas in order to combine them. The article was written in 2011, and clearly there have been some improvements to Avro since then.
A better (I’m not sure if it’s the best) way to do this, assuming you don’t want to or can’t use .avdl files, is to parse your various files into the same Schema.Parser object. It will give you a map of type name to Schema object:
List<String> schemaResourceNames = Arrays.asList("avro/foo.avsc", "avro/bar.avsc");
Schema.Parser parser = new Schema.Parser();
for (String schemaResourceName : schemaResourceNames) {
try (InputStream schemaInputStream = classLoader.getResourceAsStream(schemaResourceName)) {
if (schemaInputStream == null) {
throw new RuntimeException("Resource not found " + schemaResourceName);
}
parser.parse(schemaInputStream);
}
}
return parser.getTypes();
Thank you for this post, this was super helpful. I new there had to be a better way of including multiple schemas without using string replacement. Unfortunately, the Avro documentation is not very clear about this.
I’m glad you found it helpful!