#52 Added custom types for yaml fields to preserve newlines
Closed by psabata. Opened by mcurlej.
mcurlej/modulemd issue40  into  master

Download 52.patch

Hi All,

i've added 2 new types for the properties in the YAML file so the correct formating and newlines are preserved between loads and dumps. The issue is https://pagure.io/modulemd/issue/40. The PyYAML module will preserve the newlines only when you define the properties you need to preserve as new types. Those types will be than identified by PyYAMLs Dumper as special types with specific options which can be defined in the representer function. I haven't written any tests as the existing tests are covering my use case (according to coverage). I didn't know where to put the new types so i put them in file util.py. I'm open to suggestion if the locations is no good.

Rad! Fixing this will be excellent.

Before getting too far into review, can you add a test that exercises and reproduces the bug. We can then use this to show that the patch does or does not fix it.

rebased onto 814ca64062b3f640a91a19483f190372c624d0db

@ralph added test

rebased onto 16cfc5e55318d424dc434f931469f7f8a6f29689

Well, :+1: from me.

@sgallagh, @psabata, any input?

This test only validates that the number of newlines between the default exporter is different from the ones from the new exporter. That doesn't necessarily mean that the new exporter has the correct value.

@sgallagh is right.

A good test to add would be to:

  • Set a mmd.buildopts.rpms.macros with newlines.
  • Encode it with dumps(). Decode it back.
  • Verify that the decoded macros matched the value you sent in (newlines in particular).

:100: if you can verify that such a encode->decode step produces incorrect data before the patch in this PR.

I will try this out tomorrow and provide some feedback. I think I need to see how this works. Just glancing at the code makes me feel a bit uneasy.

Alright, so following our discussion on IRC today and myself learning some more about how YAML scalar formatting works, I'm going to close this PR because there's no actual bug here. Thank you all for your time spent on this.

YAML allows for two main scalar styles -- folded and literal. Folded style, denoted by either >, >- or quotes (!) generally ignores newlines unless there are two of them in succession, in which case they get translated into one. In the literal style, denoted by | or |-, all new lines are preserved.

What's happening here is that the dump functions use the folded style and it appears as if the newlines are doubled. While they technically are, this is just YAML representation -- when you load them back in, the string is fine and looks exactly the same.

There's no issue here and therefore no need for a code change. I will try to reproduce the original MBS component issue which, if still present, must be caused by something else.

Pull-Request has been closed by psabata

Metadata