Add custom flattening that ignores array

[trueleo]

Description

Removes rust_flatten_json because it flattens arrays as well which is not required.

Flatten JSON objects only, This includes object within an array as well. Any nested array withing an array will throw an error. This behaviour might change in future.

Changes

• Add flatten.rs module in utils
• Replace usage of rust_flatten_json with new module
• Improve schema validity check to account for array.

---

This PR has:

• been tested to ensure log ingestion and log query works.
• added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
• added documentation for new or modified features or behaviors.