Tuesday, March 20, 2007

Evaluation time

It's evaluation time. It was a real mix of experiences with some people taking to Breeze pretty well, others not - like any technology I suppose. Here's a summary of what we found out:

Breeze/Connect is a useful tool, but a fairly complex one which requires training, practice and support.

"High latency" (slow response times) during October – November 2006 (you know it becuase you get the dreaded "orange light" in Breeze) meant that Breeze/Connect ran slowly and the audio dropped out. This problem was largely fixed by David Mallyon of ITD by December 2006. However the problem led to trial participants favouring traditional teleconference for the audio component of the meeting over VOIP (Voice Over Internet Protocol). Given the response times have improved, we can now be more open to using VOIP audio, but always with teleconference as a contingency.

Positive experiences among CLI staff were due to:
o keeping things simple, not attempting to use too many features in the first few meetings
o separating the role of meeting chair and technical operator, at least until the chair becomes more confident
o for ‘high stakes’ meetings like reference groups, doing a ‘technical rehearsal’ of the meeting beforehand
o a feeling that the tool enhanced communication and teamwork, eg being able to work on a document together, being able to look at a proof of concept together, being able to see a stranger from a remote location
o a feeling that the meeting organiser controlled the technology and the process – we didn’t have to fill out a form or request work from another section to get things done.

Negative experiences among CLI staff were due to:
o dissatisfaction with vendor-supplied training
o feeling of being daunted when faced with the large number of Breeze/Connect features
o problems with network response times / VOIP audio dropping out
o potential for audio feedback loop when a face-to-face group converses with a remote participant
o using a wireless connection to the network from a laptop
o difficulty finding a quiet place in CLI where from where you can chair an online meeting – open office is noisy, meeting rooms are often all booked, sometimes the only meeting rooms available don’t have a data port.

Feedback from a sample of reference group members who participated in Breeze/Connect meetings was positive. This sample came from meetings that ran smoothly with no technical problems, just lucky I guess.