Draft Rules Feedback

by

Mon, Jun 13, 2011

Draft Rules, General Information

First off, we’d like to extend our appreciation to everyone who has taken the time to read the draft rules so far, and particularly to those who have taken the time to submit their feedback and comments on this site and via email.  Trust that all your concerns are being considered and reviewed by the judges.  We look forward to your continued contributions through the end of the draft rules review cycle (which ends on Friday 17-Jun).

We know you understand that this draft rule set represented weeks and weeks of thought, consideration, discussion, and hard work on the part of the challenge design/judge committee.   The group knew they could not go much further without hearing from those likely to participate in the competition which is why we published the draft rules.   With your input, we plan to make the final rule set as comprehensive, clear, and fair as possible.   However, simply put, we will not be able to make all the changes that have been suggested.  We will put significant thought and attention into every decision we make and final rules put in place.

In the mean time, there are some general concerns that we can address below.  The wording on any of these statements isn’t final… as necessary they will be placed in the rules in their final form:

  1. The final rules set will be in a single document.  We made it clear why it wasn’t posted in a single document.  It was not to make it more difficult for anyone, but to prevent misunderstandings and lost time on anyone’s part later if they are working with an incorrect set of rules.
  2. We have no intention of running multiple robots on the course unless necessary.  However, in order to ensure we are not limiting the number of teams who can even be interested in the event, we need to have a plan for multiple robots.  Our goal will be to let teams know in advance of the competition how likely it is that we will have to run multiple robots on the course (weeks/month(s) in advance).  It’s a goal, not a guarantee.
  3. In the case of multiple robots: a rule will be added about robots intentionally interfering with other robots (visually, electronically, or physically)
  4. In the case of multiple robots: every effort will be made that each robot will run with the same number of robots.  For example, if the schedule is such that one run needs to have two robots on the course, all runs will have two robots on the course.
  5. In the case of multiple robots: there will never be more than 3 robots on the course at the same time.
  6. Each robot has their own starting platform to return samples.  They will be marked in a clear and obvious way known to competitors well in advance of the competition (i.e. in the final set of rules presumably).
  7. The documentation section currently says that source code/schematics are required in order to help the judges with inspection.  It’s almost inevitable, with $1.5M on the line, that judges will be pressed to prove the legality of highly successful robots and we wanted them to have all the tools necessary to do that.  However, we understand providing this information is a concern to some.  Know that anything you develop for this competition (IP) will always belong to you and we have never had the intention for anything different.  We can and will work this out.  You should understand that our intention with this rule is fairness and legality of competitors ONLY.
  8. No samples will be placed in water, nor will a robot be expected to enter water for any reason.
  9. No samples will be buried.
  10. Everything you provide that is going to run with your robot is included in the weight.  This means your batteries are included, but any competition-provided required payload is not.
  11. Teams will not have access to their robot to make any modification (including any computer resets) from impound at Level 1 to the end of all Level 1 competition.  At that time, all teams will have access to their robots for some minimum period of time.  Any teams competing in Level 2 will then be impounded for Level 2.
  12. Given the shorter time of Level 1, robots will not be expected to rove the entire area in search of the sample.
  13. After the final rule set is published, teams may ask clarifying questions of the judge committee.  It is our goal that everyone shows up to the competition legal to compete, and that no one is surprised by an interpretation of the rule on the day of the event.  Teams may publicly ask questions on-line, which may be clarified for all in the FAQ.  Additionally, we will work on structuring a system so teams may choose to submit any portion of their 6-month documentation to the judges in advance if they have a concern about the legality of a technology or process that they do not want addressed in a public setting.

Thanks!  Keep the feedback coming!

17 Responses to “Draft Rules Feedback”

  1. avatar
    Danh Trinh Says:

    Could you’ll discuss the fuzzy issue of what types of “indirect” government funding is allowed to be used by the teams? For example: I am using an MIT Fab Lab to build parts for my robot. This Fab Lab was funded with money from the NSF. I think this is ok since it’s a lab open to the public for generic use but I want to be sure.

    I also saw video on the internet where similar issues were raised by the audience to Andy Petro surrounding the Nano satellite contest. Someone asked if it was ok to consult with a NASA employee which sounded very “iffy” to me in terms of contest legality.

    Reply

  2. avatar
    Michael Says:

    I’m happy with the draft rules feedback, particularly clarifications regarding multiple robots on the field and the status of our intellectual property, but some other rules need some work. Following are my final thoughts on the draft rules and recently posted clarifications.

    Disallowed Technologies:

    Don’t go overboard with the disallowed technologies. NASA’s goal is to foster innovations in autonomous navigation of varied terrain, as stated on their challenge page. It is not about material sciences.
    DC motors with cooling vents should be legal: NASA contractors have plenty of experience manufacturing motors and other things to operate in thermally sensitive environments. This challenge is not about reinventing that particular wheel, so to speak.
    Speaking of wheels, pneumatic tires should be legal: NASA contractors have plenty of experience manufacturing aluminum wheels for use in space. Most people could get ahold of aluminum wheels, or build them, but pneumatic tires are more readily available and commercially produced tires would be more reliable.
    Any part of the robot’s structure or mechanical operation should be legal if a space-rated version already exists. This covers things like the lubricant in motors and grease between gears. NASA contractors know how to do this already, space-rated parts exist, but contestants should not be forced to acquire them.
    Stick to the spirit of the competition. Not all competitors (myself, for example) will have access to University material labs and University funding and the large teams which come with them. What we have to offer are clever ideas relevant to the challenge at hand. Please do not exclude us with undue financial-and-technical burdens.

    Environment:

    Water: robots will not be required to enter water for any reason, but are bodies of water- puddles, streams, drainage ditches- present in the search environment? Whether we have to enter water or not, its mere presence forces us to either A) design a water-resistant robot or B) program our robot to recognize standing water. Both are doable, but the effort could be put towards other things if it is not necessary.
    Vegetation: I can’t think of any common rocky formations with properties quite like trees. Trees have a very narrow base, but cast a very large shadow. Trees will make orbital-resolution images less useful than they would be in an actual scenario. My biggest concern is grass or other ground vegetation. Almost any sample would be lost within grass. Worse yet, officials encouraged to place samples near rocks or trees might mistake this activity for “hiding” the samples, and what better place to hide a tiny puck than within tall grass? Reading through the draft rules gives me the impression of a grassy meadow or disused athletic field. Finding appropriately barren terrain may be difficult, but a concerted effort should be made to do so. I’d go so far as have challenge personnel comb the environment for any earth-life that does not belong.

    That is all I can think of for now. I look forward to seeing the final rules.

    Reply

  3. avatar
    Mark Curry Says:

    The issue of trees is a tricky one. Trees can cast long shadows of course, but long shadows can frequently occur on the Moon, or Mars due to the lighting conditions. On the other hand (or gripper), trees do make great landmarks, but they would never exist on a real mission. Nor would grass.

    Without a reasonable set of detectable landmarks, the teams may have to spend a lot of time developing sensors for fuzzily defined objects. By the nature of the competition, we will be operating with limited information on the environment, so that will be an issue. The issue of landmarks in a real mission would be a major concern, if for example, the robot was instructed to travel to certain locations where other, perhaps overhead, sensors had determined something interesting might be found.

    It would be great to see some measure of realism in the environment so that the developed techniques would be reasonably reusable for the intended applications.

    Cant wait to see those final rules.

    Reply

  4. avatar
    Terra Engineering Says:

    It would be nice to clarify the rules about funding sources and design sources. Even though most of us thought it was obvious in the Regolith Challenge that a company could not use a design that NASA paid them to develop, it still occurred.
    If any mechanical or software component was funded by the government in any way, the component must be obviously available to the general public before it can be used. That usually means full design access or full code access without having to jump through unusual hoops. For example, you cannot use an object ID algorithm developed in a university lab with federal funding unless every line of code can be seen by the public. Just claiming that the general public could file a freedom-of-information request to get access is not enough.
    If government funded research was used to create a product that the general public can purchase, that is fine. However, if that product is offered exclusively to one team and not available to the other teams, that is not legal.
    This stuff should be obvious, but, with money on the line sometimes people’s lines get blurred.

    Reply

  5. avatar
    Bill Says:

    I just wanted to echo what others are saying: Please don’t handicap teams of amateurs with great ideas, but minimal funding. I work in a company that sometimes works for NASA and am very aware of the extreme prices and lead times behind these components. Somewhat tongue-in-cheek: if you’re going to really limit to “real space hardware” we should all stick to very old, but space qualified, processors and then it will be a REAL challenge to do anything at all. I feel that it is in NASA’s best interest to keep the price of components low so that teams can focus on more pressing issues such as machine vision and locomotion.

    Along the same lines, I’ve been working to gather a team of amateurs. As many already know, it’s not easy to get people to dedicate time to things like this when important things like families come first. Other than time the number one hesitation I hear is cost. We really need components to be as dirt cheap as possible. I feel that us “older” and “established” engineers can bring a lot of experience and know-how to the problem. It’s worth making the barrier to entry for this group of people low.

    As I said in a previous post I certainly appreciate the time and consideration you’re giving the problem. I’d still love to see one more round of draft rules released, but if you’re short on time then we’ll make do.

    -Bill

    Reply

  6. avatar
    Danh Trinh Says:

    “In the case of multiple robots: a rule will be added about robots intentionally interfering with other robots (visually, electronically, or physically)”

    There can be visual interference that is “unintentional”. For example: a SICK S300 laser scanner comes in a stock bright yellow color. If mounted in a highly visible area such as the front of the robot another robot could mistaken it as the yellow rock sample.

    Many robots could be plastered with giant colored sponsor logos. These large colors could again be similar to the sample colors and create unintentional interference (imagine a bright green John Deere logo).

    Please think hard about a less ambiguous rule to deal with this issue.

    Reply

  7. avatar
    Danh Trinh Says:

    I’ve been to a lot of robot competitions where some teams that registered will withdraw from the final competition because of technical and\or confidence issues.

    Can you detail the consequences of withdrawing? I actually think withdrawing is a good thing. There have been too many autonomous competitions with high media expectations that ended in a boring / disappointing story due to unprepared teams bringing their robot duds.

    How are IP rights dealt with from a team that withdraws after already submitting code \ electrical schematics? Will NASA still hold valid the option of licensing?

    I personally think there is too little time to flip flop between design strategies that involve a lone robot on the field or multiple robots on the field. We need to know this at least 6 months prior. My strategy is to focus on what’s needed to win level 2 assuming multiple robots are NOT on the field. If I bet wrong then that’s a reason for me to withdraw. However, I know that the IP I submit for the “one” robot solution could still be of value to NASA.

    Reply

  8. avatar
    Michael Says:

    Final, final thoughts.

    Jack Buffington brought up the potential problem of bright corporate logos on robots when multiple robots are operating on the same field. He also mentioned the issues with rain and weather. Danh Trinh recently mentioned withdrawal from the competition, and the potential for trouble if teams are not given enough lead time on whether multiple robots will be operating on the same field or not. I can’t help but feel the same concern over these issues.

    Corporate logos add yet another layer of complexity to the single vs multiple simultaneous robots issue. Sponsors are going to want their logos visible. I don’t feel like trying to think this issue through because of how complex it has become. You have multiple robots looking for obstacles and seeing each other, potentially adding impassable terrain markers in memory where the terrain is perfectly passable; laser range finders interfering with each other; visual systems mistaking brightly colored parts of robots for samples; visual systems becoming confused by multi-colored corporate logos; visual systems becoming confused by bright, colorful hazard lights on other robots; and so on. Even with many months advance warning, having to deal with these and other unforeseen complications would be an enormous task. Each robot will be run with the same number of robots, you say, but what of the different number of logos per robot? How will you control for that? This is an issue NASA would not encounter, and it is one you can easily avoid.

    The simplest and “fairest” solution is to keep the competition to one robot at a time. If you have more teams than you expected, add more days to the competition. In the end, it may be necessary to just take the time to do this right.

    As for rain or weather, if I had to go during adverse weather I would demand that all other robots run in identical conditions. If this did not happen… well, as you say, there’s a lot of money and work on the line, and legal action starts to enter the mind of the slighted contestant. It will be necessary to add days to cover for bad weather. I’m perfectly fine with “camping out at wpi” a little longer if it means running in fair conditions. This comes back to the concept of taking the time to do it right.

    Let’s keep things fair. Let’s keep things fun.

    Reply

    • avatar
      SpaceMiners Says:

      right. that’s not gonna happen. that’s like saying, “if i’m a golfer and i shoot into a gust of wind, then everyone has to shoot in a gust of wind.”

      as you all might already know, the SpaceMiners had problems with thier 2nd generation prototype space elevator due to heavy wind conditions. see http://www.youtube.com/spaceminers

      also, many teams will be flying in and will have a strict schedule. everyone can’t just hang around and camp out. rarely do events last more than 2-3 days. it just gets too expensive for everyone :(

      Reply

  9. avatar
    Danh Trinh Says:

    When and how will WPI determine the final location of the samples? Is it the intent of WPI to bias the samples locations based on the strengths and weakness of all the submitted robot designs in an attempt to “level” the challenge difficulty for different design approaches?

    Reply

  10. avatar
    Steve Isakson Says:

    Your comment 8 indicates “nor will a robot be expected to enter water for any reason”. This may be nit picking, but as you do indicate that it may be raining during the runs, I think you mean that the bots will not be required to enter a significant depth of water. Of more significance, I assume you are not expecting to run the course on a flood plain, but I assume that minor creeks (trickles) could form during rain and the bots might have to cross these during rains. I only establish this to address traction on muddy treads and slopes.

    Reply

  11. avatar
    David Smith Says:

    Are there any limitations on who can compete and who can sponsor a team? Will someone who works at NASA be allowed to compete or someone who works for a company that has contracts with the government. Allot of local engineering companies work on government funded projects, will their employees be excluded from being on a team and their company be excluded from sponsoring a team. Will WPI students be allowed to compete or graduates from WPI. It is my belief that everyone should be allowed to compete after all isn’t the point of this competition to stimulate further research in the area of robotics. This area should get clarified in the final rules because it would be a shame for the winning team to not be able to receive the prize money because they had a conflict of interest of some kind.

    Reply

  12. avatar
    Noah Zemke Says:

    Not to beat to a dead horse but the multi-robot course brings about to many variables that make the course diferent for each robot group on the field.

    Also when it comes to logos I can tell you now that my sponor stipulates that I have thier logo on the side of my robot, and thier logo is a bright solid red.

    Thank you for taking the time to read our concerns.

    Reply

  13. avatar
    Michael Says:

    I will join Noah in the ritualistic beating of the dead horse. Multiple robots on the course will add too many variables for you or us to control.

    Reply

  14. avatar
    Bill Says:

    In terms of multiple robots, maybe you could come up with a transmitter/receiver box so that we can easily plug in the location of other robots on the field. It’s probably easier to let us run one at a time, but if we really had to run multiple robots perhaps you could come up with something clever to let us know real time what and where other robots are.

    Reply


Leave a Reply