As you said, the only rule here is that it has to last the life of the garment. That doesn't mean the consumer can't remove it if they choose to - what they do with the label is their business. If you make it easy for them to remove it, that's fine. So long as doing so doesn't prevent the label lasting the useful life of the garment, you're still within the guidelines.
Note, I haven't seen anything specific on this from the FTC either - I just can't see any reason that if a label meets all of the FTC's regulations, that it would be ruled... to not meet all of the FTC's regulations
