How to anonymize your CHR paper?

,
1

Hi there!

As the deadline for CHR papers is coming closer, I was wondering about how to address anonymization, which is implied by the double-blind review process. I suggest to make clear that anonymization is expected, as I assume not everybody who is interested in CHR will be aware of this …

I also would appreciate if you can provide some basic guidelines on how to approach anonymization … in my experience different conferences/workshops treat this in very different ways. I myself know about guides from CHI and UIST that could be used as a source of inspiration. Both are top tier conferences in the HCI domain, but I assume they are also applicable for CHR. Via Twitter, @marieke suggested ISWC as another guideline.

I think there are at least three areas where anonymization is relevant:

  1. institutions / project affiliations
  2. self-citing of previous work
  3. referencing code / data in repos

I think point 1 is well covered in the CHI guideline, including hints on the “acknowledgments” section as well as proper naming of files:

Authors are expected to remove author and institutional identities from the title and header areas of the paper, as noted in the submission instructions (Note: changing the text color of the author information is not sufficient). Also, please make sure that identifying information does not appear in the document’s meta-data (e.g., the ‘Authors’ field in your word processor’s ‘Save As’ dialog box). In addition, we require that the acknowledgments section be left blank as it could also easily identify the authors and/or their institution.

As for point 2, I really like the UIST guideline, as it distinguishes papers with significant overlap and partial overlap:

Reference Your Own Prior Work Without Violating Anonymity: UIST has explicit rules how to refer to your own prior related work.

  • if the previous own work has no significant overlap, then talk about it in 3rd person and include a full reference with names
  • if the previous own work has significant overlap, then talk about it as ‘in our prior work’ and anonymize the reference (remove the names but include the title and publication venue), in addition, submit an anonymized version of the previous work as supplementary material in PCS.

I think point 3 is a little more difficult :slight_smile: Via Twitter, @melvin.wevers suggested the following:

I often mask repos until after review or provide them upon request.

I have done this before, but what to do if you write a paper on a piece of software that is already publicly released via Git? I think creating another anonymous Git is kinda awkward …

@marieke also suggested to upload supplementary material to easychair:

Easychair also allows supplemental materials, so you can zip your code and upload it with your paper. If you do so, please add ‘see supplemental materials’ in your manuscript to refer to it, instead of ‘anonymised github repo’ so your reviewer knows where to find it :wink:

This also sounds like a great option for some cases, but what if you present a huge dataset, that can only be shared via Zenodo etc.?

I would be really happy to see some more ideas here on the issue of anonymization, so people can incorporate them in their papers until July 27 :wink:

Best
Manuel

2 Likes
2

Thanks for these thoughts and pointers @Manuel_Burghardt

Regarding point 3, one other option could be to place the code temporarily on an anonymized repo and then transfer it prior to publication.

In terms of sharing a data set, I often wait with sharing until after reviewing, since the reviewing could make you want to change the data set. One option is to share a sample of the data, or a mockup of the data.

While perhaps not feasible for this deadline, I think it might be useful to flesh out this topic and come up with a set of guidelines that we can share.

1 Like
3

As an addition to issue 3 – how to anonymize Git repositories: I found this and it seems to work really well:
Anonymous GitHub

3 Likes
4

I have big issues with the link provided, and would definitely advise anyone to not use it: the project seems very good, but it currently requires writing rights on all repos (public and private) to be able to work. Even if the developer might have good intentions, this oAuth authorization is a big security hazard from my point of view.

2 Likes
5

Here are instructions on how to anonymize data submissions with Zenodo. I saw this method in action in a CHR paper I reviewed. It works.

2 Likes