SHA checksum validation

[kurzum]

Add a CLI option to validate file integrity.

• Check the SHA sum of downloaded files against the Databus metadata
• If no SHA value is provided, skip validation but continue the download

[shivansh31414]

Hi! I'd love to work on this feature. Could you please assign it to me?
My plan is to:

• Extract SHA256 metadata during the download process
• Compute the hash of each downloaded file locally
• Compare the two and raise a warning if there's a mismatch
• Add a CLI flag like --validate-sha to toggle this behavior

Let me know if this approach works or if you'd prefer something different

[spyke7]

Hi team ! I would also like to work on this feature

[Integer-Ctrl]

Hi @shivansh31414 thanks for your interest!

Yes, your plan sounds good. Adding a toggle such as --validate-sha would be a good approach.
Maybe this could be more advanced:

• --validate-sha off → no SHA validation
• --validate-sha warning → report mismatches as warnings but continue (default if cli option not specified)
• --validate-sha error → stop execution if any file has a checksum mismatch

That would give users flexible control depending on their workflow.

[Integer-Ctrl]

@spyke7 I think it's sufficient for one person to handle this issue, but we really appreciate your interest. Thanks!

[Integer-Ctrl]

User no longer working on issue. Reopened.

[agrim-git-hub]

Hi! I'd Love to work in this issue and start on my open source journey. I haven't used these libraries in python but I have coding experience and recently only have developed a full stack ML and Generative AI app using RAG and Scikit learn. I had built a pipeline using Cohere's free starter API for students. I trained and ensemble model and then I used vector search and vector databases for personalized answers. After i completed my full project i had containerized it using docker and then created its permanent image. I believe using docker for my project has given me some rough idea on how deploying works and that's why I want to help with this issue. If i could just be guided with what exact libraries I would need to read the documentation, for this repo, I believe I can fix this issue. I'll be glad if i could get some guidance. Looking forward to your reply!

[Nama21yo]

Hi @Integer-Ctrl is there anyone who is assigned for this issue. Can I work on it?

[Integer-Ctrl]

Hi @agrim-git-hub! Great to see that you want to continue your journey here
However, the skills and interests you listed aren't quite needed for this issue.

The task is to check whether the downloaded files have a SHASUM available on the Databus and, if so, to compute the SHASUM locally and verify it against the value provided by the Databus.

We don't have the resources to guide tickets in great detail. Ideas, drafts, and questions are very welcome, but providing a step-by-step guide would be quite time-consuming.

Since you were the first to show interest, I'd wait to see whether you still want to proceed with this ticket. Otherwise, I'll hand it over to @Nama21yo if still interested.

[Nama21yo]

Yes @Integer-Ctrl am still interested and if this one is taken no worries I Will contribute on other issues. One more thing if possible.could you add me to the slack channel for more discussion the invitation inside the website have expired.
Here is my email mataniayohanes@gmail.com