Topic #2: Beta testing plan

[ihrsetrdr]

The betateam has always been open, but people often don't know that or think to join.

However, there's lots of useful information there. I wonder if there could be a scheme were we do a hybrid of the completely open and the current scheme. For example, how about a forum which is read-only by everyone and only writably by beta team members? That would be completely transparent, but wouldn't put a lot of junk info there.

I think that Vijay will have the mods make the Hidden Forum partially or fully viewable in "read only" mode, for general forum members. As one commented, this is easily reversible if going open proves to be ill advised.

 

[ChasR]

Transparency is one of the best means to accomplish nothing, by leaving oneself open to be inundated with uninformed opinion. Those of you that frequent the FF Know there are quite a few misinformed individuals that are quite vocal. Locking them out of posting in the beta forum isn't going to keep them from posting. When they see things like ppd from the earliest A3 client, which was way too high, what's going to happen? For sure they won't remember all the WUs folded for no credit because some absent minded professor forgot to set up psummary correctly.
To shorten my op-ed, so I can go put a coat of urethane on a piece of wood before bedtime, I think transparency will expose too many warts. I'd leave it like it is. Perhaps allow the beta testers a bit more freedom to discuss what they do and what is coming.

 

[ihrsetrdr]

Transparency is one of the best means to accomplish nothing, by leaving oneself open to be inundated with uninformed opinion. Those of you that frequent the FF Know there are quite a few misinformed individuals that are quite vocal. Locking them out of posting in the beta forum isn't going to keep them from posting. When they see things like ppd from the earliest A3 client, which was way too high, what's going to happen? For sure they won't remember all the WUs folded for no credit because some absent minded professor forgot to set up psummary correctly.
To shorten my op-ed, so I can go put a coat of urethane on a piece of wood before bedtime, I think transparency will expose too many warts. I'd leave it like it is. Perhaps allow the beta testers a bit more freedom to discuss what they do and what is coming.

Makes sense, especially because that seems to already be occurring in some quarters, anyway.

 

[Audioaficionado]

I don't think beta testing should be opened up willy-nilly. Only the proven dedicated experienced folders. It would only degrade the signal to noise ratio overall. Even if the vocal ones couldn't post in the threads, doesn't mean they wouldn't post else where, even off the forum and that would be very hard to control errant information from propagating like spot fires.

 

[ihrsetrdr]

Here's the latest comment made on this topic, by Bruce(site admin):

1) He is concerned about some of the information that may be contained in some of the threads. Apparently there is some information in regards to servers and flags that only beta team members have access to at this time and he feels that this information should not be let out to the general public. Would there be some sort of review of the threads before they are opened up so that sensitive information isn't revealed? Are there certain threads that might never get opened to the public due to their content? Is this a big concern for anyone else that is on the beta team?

There are a couple of pieces of information that should stay hidden, but they're not particularly important to the discussions that would be moved to a read/only forum. Moving them would be a manual process for a Moderator, and searching the topic for that information would become part of the process of declassifying the topic.

Specifically, how to configure your system to do beta testing -- we don't want people beta testing unless they understand what's involved. Moreover, we need people to discuss beta topics in the beta forum. That means keeping something like the "joining" process that we have now and having a place that allow for a free flow of information between beta testers and project owners without things getting splattered all over the website.

Once a project completes the beta process, it's either released to advmethods or, in a few instances, is withdrawn entirely and replaced by something else. Procedurally, releasing a project to advmethods could be coordinated with transferring the topic to the betateam-archives (or whatever we decide to call the read/only topic).

2) He feels that some members may be inclined to hold back a bit in their posts if they know it will become public at some future date. I can see how that would be the case in some instances, but I'm not sure how valid this argument is in regards to the beta testing team. Is this something that any of you can see becoming a problem for the effectiveness of the beta testing program?

I don't know, but I doubt it. In any case, before we move forward on a plan like this, I'll be sure to ask the betateam if they see this as a concern. Thanks for that input.

Also, I have posted the link to the beta team sign-up thread to let other teammates know that it is always open for new members to join. Hopefully this increases awareness about the beta team and some of what they do.

Good.

 

[Audioaficionado]

I never joined the beta team as I feel unqualified. I don't always have the time to carefully analyze my results and contribute useful discussions on various projects and issues.

I was a member of the FAH forum, but somehow my account got nuked and no records of my user name remain. Now it looks like I'm a n00b again as I only recently re-joined and haven't posted much since.

 

[ihrsetrdr]

Steve,

Stanford doesn't expect us(as beta testers) to do their work, but just to set up and run the beta WUs & clients...cores. I try to "earn my keep" by running the betas and reporting the results. There are only a small handful of beta testers that can analyze and articulate clearly, in discussions with the Pande staff.

I do know what you mean, I don't have as much time as I'd like to, to digest & analyze their work; I feel good about myself when I just manage to do enough reading to keep abreast with current developments.

Sign ups for beta team:

http://foldingforum.org/viewtopic.php?f=2&t=10364

 

[dfonda]

Steve,

Stanford doesn't expect us(as beta testers) to do their work, but just to set up and run the beta WUs & clients...cores. I try to "earn my keep" by running the betas and reporting the results. There are only a small handful of beta testers that can analyze and articulate clearly, in discussions with the Pande staff.

I do know what you mean, I don't have as much time as I'd like to, to digest & analyze their work; I feel good about myself when I just manage to do enough reading to keep abreast with current developments.

Sign ups for beta team:

http://foldingforum.org/viewtopic.php?f=2&t=10364

I am in the same boat as Steve , Tim as far as time. What OC is needed to be successful on the Beta Team. As of right now I am 2 hours behind for the bonus on a bigadv. Thats a 920@ 3.4GhZ. Next week I am rebuilding my 2nd rig with an i7950. That one should be better at the harder bigadv.

Would you suggest it , as something I should try?

EDIT: After reading the requirements. I don't qualify. I joined the f@h forum a couple years ago and just couldn't fit in any more forums into my time schedule. Think I'll keep doing what I am doing.

 

[ihrsetrdr]

Dave,

I'm running my i7 920s at 3.6ghz and always make the preferred deadline with time-to-spare, provided that the WU folds without any major interruptions.

I completely agree, it's difficult to keep up with activities on multiple forums when one has a life. When I'm on day shift(presently) I'm pressed for time, and have limited time for in-depth online grazing. Working swing shift(nights & weekends) affords me much more opportunity to indulge in computer hobbying. <- is that a word? 'Tis now!

 

[Audioaficionado]

Please do not apply if you're new to F@h or if you aren't an active participant in this community support forum.

http://foldingforum.org/viewtopic.php?f=16&t=8

 

[ihrsetrdr]

 

[harlam357]

Wow... looks like PantherX has done a lot of work on Bruce's posts. They're all edited last by him. On a side note, I've added PantherX as a pre-Release tester on HFM. He hasn't had a chance to participate yet but was very helpful after v0.6.0 in correcting an auto-run issue that necessitated the release of v0.6.1. If anyone here has not been included in pre-Release builds of HFM and doesn't know the secret URL where I keep builds... please let me know. I'd like our group here to be included in full.

That said... if you have HFM and keep an eye on your failed WUs it's pretty easy to check logs and report and beta WU failures to PG. All they want is the log and hardware info. If anything else is required they'll ask. If I'm off base here someone please correct me. I'm not super active over there either but I try to report failures, beta or non-beta, as much as I can.

I'll reiterate what I said in the opening discussion thread. Beta testing should be for those dedicated to the project and thus membership should be on a selective basis. The current hidden forum is just fine. What I would like to see is two things...

1) Beta WUs distributed only to clients who specify a particular flag AND have an approved Passkey. This will halt beta WUs going to those donors who happen upon any of the beta WU flags (as it exists today).

2) A recommendation program. Where a beta tester can "vouch" for another folder to join the beta program. This would, I fee, help expedite the process of adding new, qualified beta testers - if one with long standing had an established means of saying, "this guy is good and dedicated to the project". I don't think you have to "hang" around FF.org all day to be an effective beta tester, as the requirements seem to describe.