0.14 The evia project: many challenges, some solutions  (Page 2/9)

Annotations

• Given that the annotations were produced “in much more detail” than originally anticipated, and in light of the problems of information overload characteristic of many online projects, is there a risk of too much material being generated either for the sake of the end-user or, more practically, in terms of EVIA’s current and future modi operandi ?
• Is the balance at present between description of the video material and analysis thereof appropriate and conformant with the project team’s intentions? (Note the hierarchy of value implicit in the Annotation Guide , where “best quality” segments are deemed to include “rich analysis,” while “good quality” ones provide “detailed and useful analysis.”)

Peer review

• Given the hostile assessment and reward mechanisms referred to in the report, and concomitant doubts about the eventual “value of the published work to the scholarly community,” is peer review strictly necessary?
• Instead, could peer review be eliminated if collectors generating new annotation content use material within the existing EVIA resource as a model and/or receive training from the EVIA team (e.g., through an online work package) on relevant annotation or description methodologies, which they would then apply independently and without rigorous vetting? (This would result in a two-tier resource, which may have practical as well as scholarly advantages; see below.)
• If it is felt that peer review should be retained, how realistic is it to expect “academic societies” to shoulder the burden, especially given the financial and cultural obstacles mentioned in the report?

Meta-annotation

• If meta-annotation is introduced, can it be explicitly segregated from what might be termed “scholarly content” (i.e., annotations produced by collectors themselves), and if so need it be moderated at all?
• Would meta-annotations employ the same methodologies and take the same form as scholarly annotations, or would another format be more suitable both practically and in terms of likely intellectual value?

Feedback

• What feedback has been received from users other than project participants (including collectors) and what actions have been taken in response to it?
• In particular, are any aspects of the annotation process deemed to be less effective than they might be, in terms of presentation and/or potential use of content?

Impact/repercussions

• The references to “other disciplines” are not precise enough to know what the EVIA team has in mind and how they plan to ensure the maximum impact of their work. What specific repercussions and/or broader applications are envisaged?

Along with the generally positive response indicated above, my core reactions can therefore be distilled as follows:

1. although a broad range of sustainability issues is usefully explored, the plan for financial sustainability as set out in the report is not convincing at least in respect of EVIA’s secondary mission;
2. perpetuating the current modus operandi for generating and vetting annotations is neither feasible nor, perhaps, desirable;
3. meta-annotations may offer a welcome “solution” to the problem of updatability referred to in the report, but obviously they could introduce new problems without careful handling (which does not mean, however, that moderation/peer review is essential);
4. the full potential of the annotations methodology is not spelled out in sufficient detail to allow readers to gauge EVIA’s potential impact beyond the boundaries of the project.