Take this for the anecdata that it is. I interviewed at Apple, referred by old Microsoft friends that worked there. As I was trying to get a feel for things before the interview, I asked about the software testing. I was told, "don't expect what you're used to at Microsoft". The reference there is from when Microsoft often had more testers on a team than devs (ah, the good ol' days). The summary of what I was told by friends, and the questions I asked during the interview, is that testers at Apple aren't the testers that Microsoft used to have. Microsoft had testers working in MS Research, researching ways to better test software. Apple, from the impressions I got, is doing good to have testers than can write "hello, world". This was from the app side of things, not OS; I don't know if it's any different on the OS side.
But since I don't work there, I have no good inside info. But just from gut feel, I don't think my anecdata is too far off the mark. Based just on the bugs made public, I just don't get the impression that there are testers at Apple whose sole reason for being there is to tear into a piece of software and break it. There was a bug a few weeks ago posted to HN that I commented on. I don't have a link without digging through my comments, but it was something along the lines of "how could a tester not find this in five minutes of exploratory testing?" This bug is similar. It would take more than five minutes, but were this my area to test I'd pick at it once in a while when I had a few minutes. As I pick at it, I wouldn't expect to find anything, but I've got a minute between builds, so instead of randomly clicking Facebook I'll randomly click this dialog. What did the dev forget? What weird state was not accounted for? Some kind of state overflow if I click the button enough times? Shove some Unicode in there, that didn't find anything; meh, maybe I ought to move o...hey, wait a minute. Did that thing just log me in as root?
But my gut says that Apple doesn't employ a lot of testers like that.
As a Tester myself, I cannot understand why this is not covered by either unit tests or behavioral tests.
Clicking dialog buttons in rapid succession is what we (should) do once in a while. Especially in core functionalities such as the login screen.
It's one of the first screens you see as a tester. And you have default usernames, be it enabled or not.
For example, I do not own an iPhone, but at work, I made a bet with my colleague (jokingly) that I could break _something_ on his phone in a few minutes.
I did not have his finger print or pin-code, so I was very limited, I even joked "I don't need that, give it here!"
Finding out I only had a hand full of options, I focused on the emergency dialer.
As any good tester would be curious about, I wanted to check the max field length, so I entered digits, copy/paste it a few times, copy/paste that string, ("wait, no limit? Not even at 1000? why?") and so on, until I noticed the interface became laggy, so of course, I kept going.
Boom, suddenly back at the login screen, tried to open the emergency dialer, but got a full blank white screen, in the meantime the phone started heating up substantially.
Since it was a new Phone (iPhone 7 with iOS 10.x I believe) and the dev getting nervous, we decided to reboot it. That fixed the issue.
(Curious if this is still an issue in iOS 11.x)
TL;DR: As a tester this simple curiosity should be in your blood, and especially covered in behavioral tests when your software has been around for 5+ years.
I got my friends Apple Watch stuck last week just by looking at it's features. IIRC it got stuck while using the "flashlight". It suddenly froze, and it took me a while to reboot it (it got stuck once more while rebooting).
All in all, it took me about a minute to break it, and around 5 minutes to get it working again. I was getting a bit nervous.
This bug isn't caused by rapid succession or whatever, it's more of a generic end to end test. In this case someone would have had to write an exact scenario that opens a settings page, unlocks it, types in 'root', no password, presses login and it should not work.
Rapid succession button clicking usually combines many different tests:
- Performance
- Usability
- Brute-force capabilities
- Error handling
- And in this case, Security, a bug where trying to log in a couple of times on the Login screen with an empty, or set, password.
A test scenario closer to this would be:
---
When I am on the login screen
And I enter 'root' in the 'Username' field
And I enter 'thispasswordisfalse' in the 'Password' field
And I press the 'Login' button '10' times
Then I should see the text 'Your password is invalid'
---
Please note that this issue is not just in the Settings page, it takes place on the login screen as well, that's why I'm shocked, it's such a core functionality, touching so many system components.
> But since I don't work there, I have no good inside info
Actually, I've been wondering why I hear less about people working at Apple than at other big tech companies. It seems everyone and their mother work at Google or Facebook, but no so much at Apple. Do they have less software engineers, or their employees are required to be more discrete?
Do they have less software engineers, or their employees are required to be more discrete?
I know but a few that work at Apple, and of those few they strike me as less forthcoming than the multitudes I've worked with and know at Microsoft. I've wondered if part of that is because Microsoft previews/pre-announces just about everything, whereas Apple (mostly, and not so much anymore) announces it when the shipping trucks show up at the local Apple store.
So the outcome from the Microsoftie is, "it'll do this that and the other, but that's all I can say right now." From a recent conversation with an Apple employee: "they make me go in a special room to use the hardware, and I can't work from home. That's all I can say."
Probably more so, last I looked, Apple has considerably fewer software employees than the other big companies.
I believe there was an article about how Apple was finding it extremely difficult to hire ML experts because of the secrecy they require. In order to mitigate that they mad an exception for their ML/AI engineers allowing them to publish papers to external journals and present at conferences.
Presumably this is why their software is increasingly shoddy. But I wonder whether it's a direct effect of poor internal communications, or an indirect effect where the ludicrous secrecy has driven away any half-decent programmers.
But since I don't work there, I have no good inside info. But just from gut feel, I don't think my anecdata is too far off the mark. Based just on the bugs made public, I just don't get the impression that there are testers at Apple whose sole reason for being there is to tear into a piece of software and break it. There was a bug a few weeks ago posted to HN that I commented on. I don't have a link without digging through my comments, but it was something along the lines of "how could a tester not find this in five minutes of exploratory testing?" This bug is similar. It would take more than five minutes, but were this my area to test I'd pick at it once in a while when I had a few minutes. As I pick at it, I wouldn't expect to find anything, but I've got a minute between builds, so instead of randomly clicking Facebook I'll randomly click this dialog. What did the dev forget? What weird state was not accounted for? Some kind of state overflow if I click the button enough times? Shove some Unicode in there, that didn't find anything; meh, maybe I ought to move o...hey, wait a minute. Did that thing just log me in as root?
But my gut says that Apple doesn't employ a lot of testers like that.