The Art of Doing Science and Engineering: Learning to Learn

Table of contents
  1. Resources
  2. Purpose of book
  3. Motivation
  4. Things great scientists do
    1. Back-of-the-envelope calculations.
    2. Understand future of your field
    3. Communicate findings
  5. Affect of simulations
  6. Growth S curve
  7. History of hardware
  8. History of software
  9. Artificial Intelligence
    1. Expert systems
    2. Can machines think?

Resources

  • link The Art of Doing Science and Engineering: Learning to Learn by Richard W. Hamming (432 pages) (May 26, 2020)

Purpose of book

  • Why some scientists do great work, why most don’t.
  • Teach the style of thinking.
    • Style cannot be “talked about” instead it has to be experienced, similar to god, truth, art, beauty, love.
  • Greatness is defined with the conception of science and engineering as public serivce.
  • Teachers should prepare the student for the student’s future, not for the teacher’s past.
    • Although future is uncertain and usually based on the past, the past is also uncertain-or even falsely reported-than is usually recognized.
    • Due to the pace of progress the future is rather disconnected from the past, example the presense of modern computer.
  • The future of science and enginnering will be more mathematical than the past.
  • You need both education (what, when, and why to do things) and training (how to do things).
  • Knowledge doubles roughly every 17 years. Over the course of 34 year career (around your peak or retirement), the knowledge would have quadrupled.
    • Focus on things that are considered fundamentals at the time.
      • Fundamentals last a long time.
      • Fundamentals can be used to derive all the rest of the field using the standard method.
    • Learn new fields (on your own) as they arise you as to not be left behind.

Motivation

  • Short term predictions are always optimistic and long-term predictions are always pessimistic.
  • In a lifetime of many, many independent choices, small and large, a career with a vision will get you a distance proportional to n, while no vision will get you only the distance √n.
    • The accuracy of vision matters less. There are many paths to greatness.
  • With only one life to live on this earth, you ought to try to make significant contributions to humanity rather than just get along through life comfortably.
    • The true gain is in the struggle and not in the achievement.
  • Believe nothing, no matter where you read it, or who said it, unless it agrees with your own reason and your own common sense.
  • Programming is closer to novel writing than engineering. Novelists are bound only by their imagination.
    • If two novelists are given the task of writing, they will come up with different stories.
    • If you ask two engineers to put someone on the moon, the solution would almost be the same, as they are both limited by the laws of physics.
  • False beliefs will mean you will not participate significantly in the inevitable and extensive computerization of your organization and society generally.
    • For example, you assume Artificial Intelligence is useless. What if this belief turns out to be false.

Things great scientists do

Back-of-the-envelope calculations.

  • Create a modal of the phenomenon and check if the numbers being presented are in agreement (use when you hear quantitative remarks).
    • For example, knowledge double every 17 years and 90% of scientists who ever lived are now alive.
    • Model knoweledge over time y(t) = ae^(bt).
    • Model knowledge as linear over number of people y(x) = mx.
    • Assume scientist lives for 55 years, and from here you can check the claim that 90% claim.
    • Further you can modal the age of scientist and knowledge doubling years as variables, and see what ratios satisty the 90% claim.
  • Reason uage. You get a good feeling for the truth or falsity of what was claimed, as well as realize which factors you were inclined not to think about. Additionally having done the calculation you are much more likely to retain the results in your mind.

Understand future of your field

  • Devote Friday afternoons trying to understand what would happen in the future in your field, both as a scientific tool and as a shaper of the social world of work and play.
  • Think about these three questions
    • What is possible? - Science.
    • What is likely to happen? - Engineering. The desirable outcome will help guide this.
    • What is desirable to have happen? - Value judgement.

Communicate findings

It is important to communicate findings in at least three forms

  1. Writing papers and books.
  2. Prepared public talks.
  3. Impromptu talks.

Affect of simulations

  • It is cheaper to do simulations than real experiments, more flexibility in testing, possible to do things that are not possible in the lab.
  • But you should occasionally still look at nature as she is.
  • Simulation is also bringing engineering closer to science.
    • Since everything can be done with simulation. Human factor is becoming more important for a better design.

Growth S curve

  • The growth of fields follow S-shaped curve. Things begin slowly, then rise rapidly, and later flatten off as they hit some natural limits.
  • Often new innvoation will set the growth of a field ont a new S curvewhich takes off from around the saturation level of the old one.

History of hardware

  • Primitive man used pebbles to compute the sum of two amounts.
  • Sand pan and abacus.
  • Arabic numerals came from India. The Arabic numerals were deemed illegal till 1400 in favor of Roman (and earlier Greek) letters.
  • Logarithm invented by Napier in 1614.
  • Analog. Slide rule was an analog device (heavily used in 1940s), used for easier multipication of numbers by using logarithm. These contained log log scales, square and cubing scales, trignometric scales.
  • Analog. Differential analyzer was the next analog device around 1930. The first iteration by MIT had mechanical integrators. The later RDA #2 had bunch of electronic interconnections.
  • Analog. During WWII electronic analog computers were used by militiary. Used condensors as integrators in place of mechanical wheels and balls.
  • Digital. Napier also designed Napier’s bones, which were ivory rods to multiply numbers.
  • Digital. Schickard was making a machine for Kepler to do add, subtract, multiply, division machine. But due to fire it did not succeed.
  • Digital. Pascal in 1600s made desk calculator for addition and subtraction Leibnitz added multipication and division.
  • Digital. Babbage (father of modern computing) in early 1800s made difference engine, for evaluating polynomials. Since most functions can be represented as polynomials, this machine was used to create “machine-made tables” for the functions.
  • Babbagee conceived the idea of more powerful analytical engine but never finished it. It was pretty similar to von Neumann design of a computer.
  • Comptometer was designed to do repeated additions and shifting, equivalent to multipication.
  • Next came modern desk calculators, the Millinaire, then the Merchant, the Frident, and the Monroe. Originally these were hand controlled and hand powered, and later moved to electirc motors (1937). THe later ones could do square root.
  • Hollerith created the first punched-card machine (1890). The person who made this did the math and found the census could not be done in 10 years, at which point the next census had to be done.
  • In 1928 IBM used cards with rectangular holes so electric brushed could easily detect presense or absense of a hole on a card at a given place.
  • In 1935 IBM built the 601 mechanical punch to do multipication and include two additions to the product at the same time. 1 multipication in 2 or 3 seconds.
  • In 1939 George Stibitz built the Complex Number Computer (relay computer).
  • In 1946 U.S. Army got ENIAC (built by Mauchly and Eckert) the first electronic computer to compute ballistic trajectories.
  • Mauchly and Eckert also envisoned EDVAC (internally programmed machine) while building ENIAC. von Neumann was consultant to the project and wrote the report and as a result got credited for the internal programming.
  • In 1946, Mauchly and Eckert gave a course, open to all, on how to build electronic computers. The first attendee to build their own was Wilkes and the machine was called EDSAC.
  • The US federal government funnded MANIAC-1 (1952), MANIAC-II (1955), MANIAC-III (1959).
  • Mauchly and Eckert created the first commercial electronic computer UNIVAC.

History of software

  • In punch card machines, plug boards were used. These are specially wired for each job. The punches on the card were used for logic.
  • Relay machines were controlled by punched paper tapes.
  • ENIAC was controlled by wiring as if it were a gigantic plug board. Then it was converted to huge racks of dials into which decimal digits of the program could be set via the knobs of the decimal switches.
  • Programming was discussed at Mauchly and Eckert before von Neumann joined them as consultant. Ada Lovelace wrote programs for the proposed Babbage analytical engine.
  • One address codes, where each instruction contained an instruction part and the adress where the number was to be found or sent to.
  • Two address codes, typically for rotating drum machines, so the next instruction is immediately available once the previous one was completed.
  • SOAP (symbolic optimizing assembly program) was available on IBM 650 machine. It was also the first example of self-compiling.
    • program A(program A) -> B1
    • program B1(program A) -> B2
    • abs(time to produce B1 - time to produce B2) = amount of optimization produced by SOAP
  • Programming was done in absolute binary. You had to write the address part and the instruction part. Octal and hexadecimal notation was used for writing these.
  • To fix errors
    • Replace the preceding instruction to transfer to empty space.
    • Write the preceding instruction in the new space.
    • Write the fix here as well.
    • Add instruction to transfer back to the main program.
  • Mathematical libraries were created to reuse blocks of code.
  • The first programming book was by Wilkes, Wheeler, and Gill for EDSAC (1951). Interpreter was part of Appendix D, but it seems the authors did not see the importance of it.
  • SAP (Symbolic Assembly Program) was the first Assembly Language. It would translate symbolic names of the operations like ADD to binary representation and use symbolic addresses as well. So you did not have to write absolute binary.
  • FORTRAN (FORmula TRANslation) by Backus and friends.
    • Programmers intially resisted using SAP, FORTRAN because they believed absolute binary was the real way.
  • We were essentially moving from absolute to virtual machines. First we got rid of actual code instructions, then the actual addresses, then in FORTRAN the necessity of learning a lot of the insides of these complicated machines and how they worked.
  • Algol was an attempt by theoreticians to improve FORTRAN. Due to the logical nature (used of Boolean logic) it failed in the long run.
  • FORTRAN is an example of psychologically designed langauges.
  • Algol is an example of logically designed languages.
  • POLs (problem oriented languages) were special langauges that could solve a specific problem. But due to the learning involved these failed.
  • In 1962 LISP came by John McCarthy and his student build the LISP compiler. It was self-compiled.
  • ADA was designed by experts and thus hard for humans to write. 90% of ADA programs were first written in FORTRAN, debugged, tested and then converted by hand to ADA program, with high probability of errors.

Artificial Intelligence

Expert systems

  • Talk with experts in a field, extract their rules, put these rules into a program, and you hopefully got an expert.
  • It dosen’t work because some things just cannot be put in words.
  • The number of rules can grow out to be very large.
  • Experts make decisions subconciously.

Can machines think?

  • AI is a variant of this question.
  • A better version of the question is “Can we write programs which will produce ‘thinking’ from a von Neumann-type machine?”
  • Why is this question important? We cannot have humans as backups for certain tasks (like driving a vehicle on Mars). People suffer from boredom, and humans are not used to seconds of sheer panic, speed of response is another problem.

Arguments for difference between machine and human

  • Machine cannot have organic parts.
    • It does not need to. It can have wooden parts.
    • If we can reprogram a frog’s nervous system to have the frog be used as a storage machine, then would it be right to call the frog a machine.
  • Humans can think and machines cannot.
    • Chemists originally held vitalistic aspect that no organic compounds can only be made by living things. But now they hold non-vitalistic view, where they believe every organic compound can be made in the lab. Of course their is no way to prove it.
    • Usually people use this argument to name things that machine cannot do. But it goes other way as well.
    • From the machine’s perspective it can do what Newton and Einstein did, which is more than what an average person can do. So essentially from the machine’s perspective we cannot think.
  • Religious view “God made Man in His image”.
    • Religions say soul it what separates man from other animals. But soul does not have material weight.
    • Another argument is God gave man special properties.
    • The argument essentially comes to defining thinking as things that machine cannot do.
  • Physics perspective “All is atoms and void”.
    • There is no essential difference between machines and humans, hence by suitably programming machines, the machines can do anything humans can do.
    • The failure to produce thinking, is merely the failure of programmers to understand what they are doing.
  • Self-awareness and self-consciousness.
    • There is no satisfactory test to prove these things exist.
    • A machine can be programmed to say “I am self-aware”, but that is not sufficient according to you.
  • People learn from experience.
    • Reinforcement learning is a counter example to this.

continue from page 88